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QUALITY OF SERVICE FACILITY IN A 

DEVICE FOR PERFORMING IP 
FORWARDING AND ATM SWITCHING 

RELATED APPLICATIONS 

This application claims the benefit of priority under 35 
U.S.C. 119(e) to U.S. Provisional Application Serial No. 
60/090,028, filed Jun. 19, 1998, and is related to U.S. patent 
application Ser. No. 09/237,128, filed Jan. 25, 1999, and 
entitled "NETWORK PACKET FORWARDING LOOKUP 
WITH A REDUCED NUMBER OF MEMORY 
ACCESSES," U.S. patent application Ser. No. 09/336,090, 
filed Jun. 18, 1999, and entitled "AN INTERCONNECT 
NETWORK FOR OPERATION WITHIN A COMMUNI- 
CATION NODE/' U.S. patent application Ser. No. 09/336, 
229, filed Jun. 18, 1999, and entitled "DEVICE FOR PER- 
FORMING IP FORWARDING AND ATM SWITCHING," 
and U.S. patent application Ser. No. 09/335,947, filed Jun. 
18, 1999, and entitled "METHOD AND SYSTEM FOR 
ENCAPSULATING/DECAPSULATING DATA ON A PER 
CHANNEL BASIS IN HARDWARE". The entire contents 
of each of the applications are hereby incorporated by 
reference. 

TECHNICAL FIELD 

The present invention relates generally to communication 
nodes and more particularly to Quality of Service (QoS) 
features in a single communication node for performing IP 
forwarding and ATM switching. 

BACKGROUND OF THE INVENTION 

QoS covers a broad range of issues in computer networks. 
QoS features can take the form of preferred service for 
particular data flows. QoS features can also include conges- 
tion control. As used herein, the term "QoS features" refers 
to those features in a digital communication network that 
provide a capability to differentiate between data flows so 
that network service providers some traffic differently than 
other traffic. The need for QoS features arises from different 
types of network traffic having different transmission 
requirements. By way of example, to avoid echoes, voice 
traffic typically requires a 64 kbs bandwidth, with less than 
100 ms of delay. Alternatively, non-interactive broadcast 
video typically requires a 271 Mbs bandwidth, but does not 
have a strict delay requirement. To be competitive, network 
service providers need to provide differentiated classes of 
service. 

In conventional systems, ATM networks have been 
viewed as separate universes from IP networks. ATM net- 
works work well for a subset of services, and IP networks 
work well for a different subset of services. Traditionally, 
ATM networks have been viewed as preferential for appli- 
cations requiring more sophisticated QoS features. For 
example, the ATM Forum has defined five service categories 
for ATM: Constant Bit Rate (CBR); real-time Variable Bit 
Rate (rtVBR); non-real-time Variable Bit Rate (nrtVBR); 
Unspecified Bit Rate (UBR); and Available Bit Rate (ABR). 
When a service provider sets up an ATM virtual circuit 
(VC), the service provider and the user contract for one of 
the service categories. With each service category, comes a 
set of transmission priority parameters that are specific to the 
category. 

However, as multimedia applications have infiltrated 
computer networking, IP QoS features have improved. 
Today, IP QoS features include the ReSerVation Protocol 
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(RSVP); the Integrated Services models (IntServ); and the 
Integrated Services over Specific Link Layers (ISSLL). 
Together, these components provide comprehensive QoS 
features for end-to-end flows, but still do not provide all of 
the QoS features available from ATM. Additionally, this type 
of end-to-end flow regulation lacks the flexibility required 
for adapting to emerging technologies. 

Given that neither IP nor ATM offer a complete multiser- 
vice solution, many service providers choose to operate dual 
networks. IP networks support applications such as Internet 
access and virtual private networks (VPNs), whereas ATM 
networks support Frame Relay (FR), VPNs, circuit 
emulation, private branch exchanges (PBX) and other appli- 
cations where reliability and more rigorous QoS are a 
priority. These dual networks can be a complex and expen- 
sive aggregation of core routers connecting smaller Access 
Points of Presence (PoPs) to the core transport capacity. 
These structures are fragile, with frequent service outages 
due to performance limitations and equipment failures. 
Enterprises cannot afford to be exposed to significant down 
time due to failures or updates associated with conventional 
technology. 

Accordingly, an object of the invention is to provide 
enhanced QoS features in a single communication node for 
performing IP forwarding and ATM switching, 

A further object of the invention is to provide QoS 
features, which are capable of accommodating emerging 
technologies, in a single communication node. 

Another object of the invention is to provide QoS 
features, which are capable of accommodating a variety of 
communication protocols, without requiring the mainte- 
nance of costly parallel networks. 

These and other objects of the invention will be apparent 
with respect to the following description of the invention. 

SUMMARY OF THE INVENTION 

The invention is directed to a facility and related methods 
for providing Asynchronous Transfer Mode (ATM) and 
Internet Protocol (IP) Quality of Service (QoS) features in a 
digital communication node. Optionally, the QoS facility of 
the invention also provides Frame Relay (FR) QoS features. 
According to one embodiment of the invention, the facility 
comprises a plurality of logical input ports, a plurality of 
logical output ports, switching elements, routing elements 
and QoS elements. 

The logical input ports are adapted for receiving input 
data flows from external data sources. Similarly, the logical 
output ports are adapted for transmitting output data flows to 
a plurality of external data destinations. According to the 
invention, the input and output data flows can be ATM-based 
data flows or IP-based data flows. The input and output data 
flows can also be IP over ATM. That is, IP packets can be 
carried in ATM cells. In a further embodiment of the 
invention, the logical input ports are included in a common 
physical interface. According to a further aspect, the input 
data flows through the common physical interface include 
Synchronous Optical Network (SONET) frames. 

The switching elements are adapted for switching ATM 
data cells from one of the logical input ports toward at least 
one of the logical output ports, along a selected forwarding 
path. According to a further feature of the invention, the 
switching elements include ATM lookup elements for iden- 
tifying toward which of the logical output ports particular 
ATM data cells should be switched. 

The routing elements are adapted for routing IP data 
packets from one of the logical input ports toward at least 
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one of the logical output ports along a selected forwarding 
path. According to a further embodiment of the invention, 
the routing elements include IP Lookup elements for identi- 
fying toward which of the logical output ports to rout a 
particular IP data packet, in response to information con- 
tained in the particular IP data packet. 

The QoS elements are common to the switching elements 
and the routing elements and provide AIM QoS features to 
the ATM data cells and IP QoS features to the to the IP data 
packets. Optionally, the ATM lookup elements are further 
adapted for determining which of the ATM QoS features 
should be applied to a particular ATM data cell. According 
to a time saving feature, the lookup elements identify a 
forwarding path and determine the applicable ATM QoS 
features in a single lookup operation. 

The ATM QoS features include one or more of, Constant 
Bit Rate (CBR), Unspecified Bit Rate (UBR), non-real-time 
Variable Bit Rate (nrtVBR), real-time Variable Bit Rate 
(rtVBR) and Available Bit Rate (ABR), and the IP QoS 
features include one or more of, Provisional QoS, Differen- 
tiated Services, and Integrated Services. 

In another embodiment of the invention, the facility for 
providing ATM and IP QoS features includes a mechanical 
housing that contains both the switching, routing and QoS 
elements. In this way, a facility according to one embodi- 
ment of the invention, provides an integrated system for 
switching ATM data cells, routing IP data packets and 
providing ATM and IP QoS features. Thus, a facility accord- 
ing to this embodiment of the invention enables service 
providers to avoid maintaining costly parallel networks; one 
for switching ATM data cells and one for routing IP data 
packets. The facility of the invention also enables service 
providers to provide different classes of service (e.g. coach, 
business and first class) for both ATM -based and IP-based 
data flows; thus, providing an additional source of revenue 
from clients willing to pay for enhanced bandwidth guar- 
antees. 

In alternative embodiments, the QoS facility provides call 
control elements. The call control elements enable the 
facility to form service contracts with client networks. The 
service contracts typically specify QoS features, such as 
bandwidth guarantees, that the communication node agrees 
to provide to data flows received from or transmitted to a 
client network. Optionally, the call control elements deter- 
mine the available bandwidth of the communication node, 
and accept or deny requested service contracts in response to 
the available bandwidth. 

Another embodiment of the invention provides traffic 
control elements. The traffic control elements interpret the 
bandwidth requirements associated with a service contract, 
and signal control information to devices along the forward- 
ing path of the contracted data flow to reserve adequate 
bandwidth to provide the data flow with the contracted for 
QoS features. 

A further feature of the invention is that the QoS elements 
are adapted to interpret the industry standard RSVP proto- 
col. The RSVP protocol is a signaling protocol for an 
external data destination to request a service contract from 
the communication node. Typically, the service contract 
applies to data flows that the external data destination 
anticipates receiving from an external data source. 

A facility according to one embodiment of the invention 
can be viewed as providing a plurality of logical functions 
along a data forwarding path. Classification, scheduling and 
policing are examples of such logical functions. Classifica- 
tion elements of the invention categorize received ATM data 



10 



15 



20 . 



25 



30 



35 



45 



50 



60 



65 



cells and IP data packets, based on which, if any, QoS 
features apply to the received cells and packets. The sched- 
uling elements schedule routing of the IP data packets and 
switching of the ATM data cells in response to categoriza- 
tion by the classification elements. The policing elements 
monitor data flows to ensure that data flows received from 
external sources do not exceed agreed upon service con- 
tracts. According to one embodiment of the invention, the 
policing elements discard data that fails to conform to an 
agreed upon service contract According to an alternative 
embodiment, the policing elements mark the nonconforming 
data. Data so marked is more likely to be discarded, should 
the communication node become congested. 

According to another embodiment of the invention, the 
QoS facility employs alternative methods for identifying 
nonconforming data and for determining which data to 
discard if the communication node should become con- 
gested. By way of example, according to one feature, ATM 
data cells make up an ATM frame and the policing elements 
include Partial Packet Discard (PPD) elements for discard- 
ing selected additional ATM data cells included in an ATM 
frame, in response to having to discard one or more non- 
conforming ATM data cells included in the ATM frame. 
Typically, the PPD elements discard the ATM data cells of 
an ATM frame, which are received subsequent to the non- 
conforming ATM data cell, except for the last cell of the 
frame. The PPD elements do not discard the last cell of the 
ATM frame because that cell includes an end of a cell 
indicator. 

An alternative approach to PPD is Early Packet Discard 
(EPD). According to a further aspect of the invention, the 
policing elements include EPD elements. According to the 
EPD protocol, the policing elements also include queuing 
elements for buffering the forwarding of ATM data cells, and 
the EPD elements discard entire ATM frames in response to 
the queuing elements reaching a selected level of fullness. 
By discarding the entire frame, the QoS facility avoids the 
overhead associated with transferring useless, partial ATM 
frames. It should be noted that PPD and EPD can be applied 
simultaneously by a policing element, and they can also be 
used at an output port for congestion control. 

According to an alternative embodiment of the invention, 
the QoS facility comprises a plurality of logical input ports, 
a plurality of logical output ports, a plurality of communi- 
cation modules, and QoS elements. The logical input ports 
are adapted for receiving input data flows from external data 
sources. The logical output ports are adapted for transmitting 
output data flows to a plurality of external data destinations. 
The data flows can be ATM-based data flows or IP-based 
data flows. 

In a further aspect, the output ports include Random Early 
Detect (RED) elements. The RED elements monitor queue 
elements that buffer IP data packets and ATM data cells. In 
response to the queues reaching a selected average level of 
fullness, the RED elements substantially randomly discard 
IP data packets and ATM data cells. In this way, the QoS 
facility of the invention avoids the possibility of a data flow 
using up a disproportionate amount of bandwidth in the 
communication node. 

The communication modules include IP packet routing 
elements for routing IP data packets from one of the logical 
input ports to one or more of the logical output ports. The 
communication modules also include ATM cell switching 
elements for switching ATM data cells form one of the 
logical input ports to one or more of the logical output ports. 

The QoS elements provide ATM QoS features to ATM 
data cells, and provide IP QoS features to IP data packets. 
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The ATM QoS features include one or more of, Constant Bit 
Rate (CBR), Unspecified Bit Rate (UBR), non-real-time 
Variable Bit Rate (nrtVBR), real-time Variable Bit Rate 
(rtVBR) and Available Bit Rate (ABR). The IP QoS features 
include one or more of, Provisional QoS, Differentiated 
Services, and Integrated Services. In a further embodiment, 
at least some of the QoS elements are distributed among the 
communication modules. 

According to a further feature, the QoS facility includes 
an interconnect- The interconnect is in digital communica- 
tion with the communication modules and is adapted for 
forwarding ATM data cells and IP data packets between the 
communication modules. In one preferred embodiment, the 
interconnect is in electrical communication with the com- 
munication modules, while in other embodiments the inter- 
connect is coupled to the communication modules via fiber 
optical connections. Optionally, at least some of the QoS 
elements are included in the interconnect. 

The QoS elements can also include a control processor for 
controlling operation of the QoS elements. In a further 
aspect, the control processor populates tables that are at least 
in part representative of the QoS features to be applied to 
received ATM data cells and received IP data packets. 
Optionally, the QoS elements include lookup elements that 
access the tables and schedule routing of IP data packets and 
switching of ATM data cells through the communication 
node, based at least in part on the populated tables. In a 
further embodiment, each of the communication modules 
include a communication module processor. The communi- 
cation module processors are in communication with the 
control processor and assist the control processor in con- 
trolling the QoS elements. 

According to a further embodiment of the invention, the 
communication modules include a queuing structure for 
intermediately storing ATM data cells and IP data packets 
transferred from the interconnect to the communication 
modules for output via one or more of the logical output 
ports. According to another aspect, the communication mod- 
ules include one or more logical output ports associated with 
each of the physical output ports. Additionally, the queuing 
structure further comprises a plurality of output queues 
associated with each of the physical output ports, wherein 
each plurality of output queues is adapted for intermediately 
storing IP data packets and ATM data cells destined for 
output through an associated physical output port. Accord- 
ing to another feature, each of the queues included in a 
particular plurality of queues has an assigned priority rela- 
tive to other queues included in the plurality. Data stored in 
queues having a relatively higher priority is scheduled for 
output in preference to data stored in relatively lower 
priority queues. Which, if any, of the ATM and IP QoS 
features are associated with the data determines in which 
queue the data is stored. It also determines the output 
scheduling of the data stored in the queue. According to 
another feature, congestion control elements are used to 
monitor the amount of data in the queues and selectively 
drop or mark IP packets and ATM cells when the queue 
occupations reach selected thresholds. 

In a further feature of the invention, the queuing structure 
includes a traffic shaping element comprised of a calendar 
queue. The calendar queue intermediately stores at least 
selected ATM data cells and IP data packets destined for the 
output queues. The traffic shaping element schedules the 
transfer of ATM data cells and IP data packets from the 
calendar queue to the output queues, based at least in part, 
on which of the ATM and IP QoS features apply to the ATM 
data cells and IP data packets stored in the calendar queue. 
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According to a further practice of the invention, the 
queuing structure also includes an output stack. The output 
stack is adapted for intermediately storing ATM data cells 
and IP data packets that are destined for transfer from one of 
the output queues. 

In an alternative embodiment, the invention comprises a 
method for providing ATM and IP QoS features in a digital 
communication node. The method comprises the steps of: 
receiving ATM data cells and IP data packets into the digital 
communication node; determining if any ATM QoS features 
are associated with a received ATM cell, and if so, assigning 
a priority for forwarding the received ATM data cell through 
the digital communication node, wherein the priority is 
representative of the ATM QoS features associated with the 
cell; determining if any IP QoS features are associated with 
a received IP data packet, and if so, assigning a priority for 
forwarding the received IP packet through the digital com- 
munication node, wherein the priority is representative of 
the IP QoS features associated with the packet; scheduling 
forwarding of received ATM cells and received IP packets 
through the communication node, based at least in part on 
the assigned priority; forwarding the received ATM cells and 
IP packets to an external destination. 

In further embodiments, the invention includes additional 
elements and methods for providing ATM and IP QoS 
features in a digital communication node. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The subject matter regarded as the invention is particu- 
larly pointed out and distinctly claimed in the concluding 
portion of the specification. However, the invention, both as 
to organization and method of practice, together with further 
objects and advantages thereof, may best be understood by 
reference to the following illustrative description taken in 
conjunction with the accompanying drawings in which like 
numerals refer to like elements, and 

FIG. 1 depicts a modular communication node including 
QoS features according to an illustrative embodiment of the 
invention; 

FIG. 2 depicts a switching shelf for use in the illustrative 
embodiment of the of FIG. 1; 

FIG. 3 depicts a channelized SONET scheme used in the 
illustrative embodiment; 

FIG. 4 is a logical block diagram of a portion of the 
switching shelf of FIG. 2; 

FIG. 5 is a logical block diagram of a line card of the type 
depicted in the switching shelf of FIGS. 2 and 4; 

FIG. 6 is a simplified block diagram illustrating operation 
of the modular communication node of FIG. 1; 

FIG. 7 is a logical flow diagram illustrating steps per- 
formed on communication traffic by the modular commu- 
nication node of FIG. 1; 

FIG. 8 is a block diagram depicting the logical compo- 
nents of a QoS facility according to an illustrative embodi- 
ment of the invention; 

FIG. 9 is a functional block diagram of a portion of the 
switching shelf of FIG. 2 and illustrates elements of a QoS 
facility according to one embodiment of the invention; 

FIG. 10 is a flowchart illustrating the steps that the 
communication node of FIG. 1 performs during input pro- 
cessing; 

FIG. 11 is a functional flow diagram illustrating the steps 
that the communication node of FIG. 1 performs during 
input processing; 



10/27/2004, EAST Version: 1.4.1 



US 6,611,522 Bl 

7 8 

FIG. 12 illustrates the logical format of a SONET frame; trative embodiment of the invention also provides a descrip- 

F1G. 13 is a more detailed logical block diagram of the uon of an illustrative embodiment of the communication 

receive ASIC included in the illustrative line card of FIG. 6; node - 

FIG. 14 illustrates the logical format of a DS-3 PLCP As mentioned above, a digital communication node inte- 

frame- s grating a QoS facility according to the invention preferably 

FIG. IS illustrates the logical format of a PPP frame; includes both an IP data packet routing facility and an ATM 

r ^ , data cell switching faculty. In this context, forwarding 

FIG. 16 illustrates the logical format of a Frame Relay refers , 0 me ?assing of data be , ween a port ^ one 

^ ramc ' or more destination ports in the communication node, such 

FIG. 17 illustrates the logical format. Of an AAL5 IDU; 10 as a switch, a router or a switch/router. "Routing" refers to 

FIG. 18 illustrates the logical format of an ATM cell. the accumulation of topology information to provide infor- 

F1G. 19 is a flowchart illustrating the steps that the mation to a forwarding table or similar structure by the 

communication node of FIG. 1 performs during ATM cell communication node that is used for directing input data 

input processing- toward a destination. "Switching" refers to the directing of 

FIG. 20 illustrates the logical format of a communication 15 ^ or u other modularized information through inter- 

cell that is used internally in the communication node of switching nodes to connect a sender with a receiver 

ma connecUon-onented environment. 

— ' . i • i -i ■ „ mi ♦ at\x Uninm The illustrative embodiment eliminates the need for hav- 

FIG. 21 is a logical diagram illustrating ATM lookup A ... , . • » j * i 

c j u *u • actp «f nr 11. me separate switching and routing networks. A digital 

performed by the receive ASIC of FIG. 13; 20 • ^ j 1 • ^ o e -r« «i- • 

r ' . . communication node employing a QoS facility according to 

FIG. 22 is a flowchart illustrating the steps performed by mc iavention can handle both ^ data cclls ^ IP data 

the receive ASIC of FIG. 13 during IP input processing; packets Tfae commumcation node may be employed in IP 

FIG. 23 illustrates the logical format of IP header data; networks, such as the internet, intranet or extranet, or more 

FIG. 24 illustrates data structures and tables that are traditional switching environments, such as virtual private 

employed during IP lookup; 25 networks (VPNs), private data networks and SNA (Systems 

FIG. 25 illustrates a logical format of a DANET structure; Network Architects) networks. The illustrated communica- 

FIG. 26 is a flowchart illustrating steps performed by the * on « ode r ^ °lwu^? ™l ip 80 ^ 

receive ASIC of FIG. 13 during IP lookup; (Synchronous Optical Network^ , the routing of IP packets 

7 , . , . P * , over ATM and pure ATM switching. More generally, the 

FIG. 27 is a diagram illustrating the indexing of a lookup 30 nhlstrative embo diment eliminates, the separation of Open 

array during IP lookup; Systems Interconnection (OSI) layer 2 devices and layer 3 

FIG. 28 is a example illustrating the relationship between devices so that layer 2 data units and layer 3 data units may 

lookup arrays and DANET structures during IP lookup; be directed toward their destinations by a single communi- 

FIG. 29 is a flowchart illustrating the steps that are cation node through a common QoS facility, 
performed by the modular communication node of FIG. 1 35 Th e illustrative digital communication node includes 
during a switching stage; input ports for receiving input data traffic from external 
FIG. 30 is a logical block diagram of an interconnect card sources and output ports for directing the input data traffic 
of the type used to transfer information between the line towards external destinations. Each input data port is also 
cards of FIG. 4; 4Q tied to a communication line, such as a fiber optic line. 
FIGS. 31A and 31B depict interconnect priority queues; Similarly, each output port is tied, likewise, to a communi- 
ty • £. 1 1 . , cation line (e .e. a fiber optic line). The communication node 
FIG. 32 is a functional diagram illustrating, among other ^ v a " r - v+ j T n 1 * 
, 4 .. ^ 0 7? f j? t ** provides an AIM cell forwarding facility and an IP packet 
output operations, QoS operations performed by the transmit £ u • . . t-u a™ h 
ASIC of FIG 5 durin out ut rocessin ■ forwarding facility for each input port. The ATM cell 
o . uring ou pu processing, forwarding facility determines, for each ATM cell received 
FIG. 33 is a more detailed diagram of the transmit ASIC « ^ ^ ^ ^ which ^ io ^ fof mtpm]ag the 
included in the illustrative line card of FIG. 5; Am ceU ^ Tp packc( forwarding facility determines, for 
FIG. 34 is a logical block diagram of the transmit and each IP packet received by the input port, which output port 
calendar queues employed on the transmit ASIC of FIG. 33 t0 f or outputting the IP packet. Hence, each input port 
for providing QoS features according to an illustrative ^ mav receive both ATM cells and IP packets and the corn- 
embodiment of the invention; and munication node will properly direct the ATM cells and IP 

FIG. 35 illustrates the relationship between the transmit packets, 

queues of FIG. 34 and an output FIFO storage element. The illustrative QoS facility is integrated into the ATM 

cell and IP packet forwarding facilities, along with the 

DESCRIPTION OF THE ILLUSTRATED 5S contro] processors that schedule the forwarding of ATM 

EMBODIMENT cells and 1P packets> xh e Q 0 S facility provides ATM QoS 

The illustrative embodiment of the invention provides a features such as Constant Bit Rate (CBR), Unspecified Bit 

facility for furnishing both Asynchronous Transfer Mode Rate (UBR), non-real-time Variable Bit Rate (nrtVBR), 

(ATM) and Internet Protocol (IP) Quality of Service (QoS) real-time Variable Bit Rate (rtVBR) and Available Bit Rate 

features in an illustrative digital communication node. In 60 (ABR). The QoS facility also provides IP QoS features, such 

contrast to prior digital communication nodes, which main- « Provisional QoS, Differentiated Services, and Integrated 

tain separate parallel networks for performing IP forwarding Services. 

and ATM switching, the illustrative digital communication The discussion below summarizes the architecture and 

node is an integrated device, which performs both IP for- operation of a QoS facility according to the invention. As 

warding and ATM switching. Since a QoS facility according 65 depicted, it is integrated within an illustrative digital com- 

to invention is preferably integrated into such a novel digital munication node for performing both ATM switching and IP 

communication node, the following description of an illus- forwarding. 
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FIG. 1 depicts an illustrative digital communication node 
10 for providing ATM switching and IP routing, and incor- 
porating a QoS facility according to the invention. The 
illustrative communication node 10 includes eight switching 
shelves 12, eight access shelves 14, two control shelves 16, 
and an extension shelf 18. The switching shelf 12 provides 
core switching functionality for the node 10. As shown, the 
node 10, optionally includes multiple switching shelves 12 
to increase the bandwidth of the node 10. This modularizing 
of the switching functionality enables network service pro- 
viders to choose the switching bandwidth that is appropriate 
for their needs. Each access shelf 14 includes a pair of linear 
terminal multiplexers that create a structured OC-48 data 
stream or individual OC-12/STM4, OC-3/STM1, DS-3 and/ 
or E3 tributaries. Illustratively, the communication node 10 
employs eight access shelves 14, thereby providing an 
access shelf 14 for each corresponding switching sbelf 12. 
Each control shelf 16 contains a redundant pair of control 
processors. One oversees operation of the communication 
node 10. The other is a standby processor. As discussed 
below in more detail, optionally, the control processors of 
the control shelves 16 include a portion of the QoS facility 
of the invention. The extension shelf 18 is a 160 Gbps switch 
for interconnecting the up to eight switching shelves 12. The 
extension shelf 18 enables data transfer between the switch- 
ing shelves 12. By way of example, the extension shelf 18 
enables an input data stream to be received at an input port 
of one switching shelf 12 and to be transferred to and output 
from one or more output ports of one or more other 
switching shelves 12. 

FIG. 2 depicts a switching shelf 12 of the type shown in 
the communication node 10 of FIG. 1. The switching shelf 
12 includes a housing 20 for containing the components of 
the switching shelf, including eight line cards 22. The eight 
line cards 22 are printed circuit boards that contain circuitry 
for receiving and transmitting data. Each tine card 22 is 
designed to receive an OC-48 input data stream, correspond- 
ing to 2.488 gigabits per second (Gbps). SONET is a 
standard that defines a family of fiber optic transmission 
rates that facilitate the internetworking of transmission prod- 
ucts for multiple vendors. The optical transmission rates are 
known as optical carry (OC) rates. The SONET OC rates are 
defined in TABLE 1 as follows: 

TABLE 1 



OC Level 


Line Rates 


Capacity 


OC-1 


51.84 Mbps 


28 DSls oi 1 DS3 


OC-3 


155.52 Mbps 


84 DSls oi 3 DS3s 


OC-9 


466.56 Mbps 


252 DSlsor9DS3s 


OC-1 2 


622,08 Mbps 


336 DSls oi 12 DS3s 


OC-18 


933.12 Mbps 


504 DSls oi 18 DS3s 


OC-24 


1.244 Gbps 


672 DSls oi 24 DS3s 


OC-36 


1.866 Gbps 


1008 DSls oi 36 DS3s 


OC-48 


2.488 Gbps 


1344 DSls oi 48 DS3s 


OC-96 


4.976 Gbps 


2688 DSls oi 96 DS3s 


OC-1 92 


9.953 Gbps 


5376 DSls oi 192 DS3s 
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As can be seen, OC-48 is one of the specified line rates. 
In the capacity column of TABLE 1, references are made to 
DS-1 and DS-3 rates. These are respective line rales in the 
DS hierarchy of digital signal speeds that is used to classify 
capacities of lines or trunks. The fundamental speed level in 
the DS hierarchy is DS-0, which corresponds to 64 kilobits 
per second. DS-1 corresponds to 1.54 megabits per second, 
and DS-3 corresponds to 44.736 mbps. 

The switching shelf 12 also contains interconnect module 
cards 24, which occupy 3 slots. The interconnect module 
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cards 24 are printed circuit boards that provide switching 
and routing capacity to facilitate communication between 
the line cards. The interconnect module cards 24 form the 
core of the "interconnect," which will be described in more 
detail below. Switch processor modules 26 occupy the 
remaining two slots in the switching shelf 10. These pro- 
cessor modules 26 manage board level status information for 
the switching shelf 12. 

The depicted communication node 10 provides a chan- 
nelization SONET/SDH mode of operation, such that each 
OC-48 fine card 22 can be configured for DS-3, OC-3 and 
OC-12 tributary configuration. 

FIG. 3 shows an example of such channelizatioa A single 
OC-48 input stream 30 has tributaries that include an 
OC-12C packet over SONET tributary 32 and an OC-12 
ATM tributary 34. The tributary 38 divides into four OC-3 
tributaries, including an OC-3C packet over SONET tribu- 
tary 44 and an OC-3 ATM tributary 46. The tributary 47 
divides into three DS-3 tributaries, including an ATM tribu- 
tary 40 and a packet over SONET tributary 42. Each of the 
line cards 22 demultiplexes the OC-48 input stream into the 
specified tributaries and then operates on the tributaries (i.e. 
"channels") separately. The configuration of the tributaries is 
software controlled and may be dynamically altered. 

FIG. 4 depicts a portion of the switching shelf 12, along 
with multiplexers 50 and 52, which are located in the access 
shelf 14. For illustrative purposes, FIG, 4 only depicts four 
of the eight potential line cards 22 that can be included in a 
switching shelf 12. The block diagram 48 of FIG. 4 shows 
illustrative line cards 53, 55, 57 and 59, an interconnect 62, 
the SONET multiplexers 50 and 52, and a control processor 

64. In operation, data enters a SONET multiplexer 50 by 
way of lines 52a-52d. The multiplexer 52 passes a single 
physical OC-48 data stream to the line card 59 by way of fine 

65. The line card 59 forwards information stripped from the 
OC-48 data stream to the interconnect 62, by way of line 63. 
The interconnect 62 processes the received information and 
forwards it to a destination line card, by way of example, 
line card 53, along line 61. The destination line card 53, in 
turn, transfers the received information by way of the OC-48 
interface 51 to the SONET multiplexer 50. The multiplexer 
50 forwards the received information to an external source 
by way of lines SOa-SOd. Information transfers involving 
line cards 55 and 57 occur in much the same fashion. 

FIG. 5 is a more detailed logical block diagram of an 
illustrative line card 59. Each of the other line cards 53, 55, 
and 57 has a similar layout. The line card 59 includes a Line 
Card Processor (LCP) 72 and a memory 74. The memory 74 
may take many different forms, including a random access 
memory (RAM) or a read only memory (ROM). The fine 
card 59 also includes application specific integrated circuits 
(ASICs) 60, including a receive ASIC 70 and a transmit 
ASIC 71. The receive ASIC 70 is responsible for receiving 
incoming data and processing the data so that the data is 
ready to be transferred over the interconnect 62. The trans- 
mit ASIC 71 receives data from the interconnect 62 and 
forwards data out over an output port to the output line 65. 
The receive ASIC 70 includes a logical QoS portion 70a, 
The QoS portion 70a provides classification and policing 
functions. The transmit ASIC 71 includes a logical QoS 
portion 71a, The QoS portion 71a provides output data 
scheduling and shaping. These QoS functions are discussed 
in further detail below. 

As those skilled in the art will appreciate, the depiction of 
the ASIC 60 is considered to be merely illustrative. In other 
embodiments, the receive ASIC 70 and the transmit ASIC 71 
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may be implemented as a single ASIC. Alternatively, the places the data in a suitable format for forwarding over the 

ASICS 71 and 70 may be implemented as more than two interconnect 62. The input processing stage 82 employs IP 

ASICS. Further, the ASICs 71 and 70 may be implemented routing and ATM switching lookups to identify QoS features 

in alternative circuitry, such as field programmable gate and a destination address. The routing/switching stage 84 

arrays (FPGAs) or in software. 5 forwards the input toward the appropriate output line cards. 

j . . ' , c - c - , As is discussed in further detail below, the data is forwarded 

As men toned above, each of the line cards 53, 55 and 57 tQ ^ ^ m ^ ^ ^ ^ 

has a similar architecture to Aat depicted in FIG. 5. Hence, ^ intcrconnect 62 . ^ output p r0CCSS i D g stage 86 encap- 

the line card 53 includes ASIC 54, tine card 55 includes &ulates ^ data feceived over ^ md ^ds 

ASIC 56 and lme card 57 includes ASIC 58, mc data out the a p pro priate output ports so that the data 

Those skilled in the art will also appreciate that the reaches the intended destinations. The output processing 

depiction of the line card 59, shown in FIG. 5 is also stage 86 also shapes and schedules the output of the data to 

considered to be merely illustrative and not limiting of the meet any IP or ATM QoS features associated with the data, 

present invention. Other line card configurations may be In case of congestion, the output processing stage 86 also 

used to practice of the present invention. Moreover, the drops or marks IP data packets and AIM data cells, 

functionality provided by the line card 59 need not be 35 FIG. 7 is a functional flow diagram 90 illustrative of data 

implemented on a line card, per se, but rather may be processing in the communication node 10 of FIG. 1. Data 

implemented in a different fashion or by a different hardware enters an input line card, for example line card 59, by way 

configuration. Additionally, the receive ASIC 70 and the of an OC-48 interface 92. A demultiplexer 94 demultiplexes 

transmit ASIC 71 need not be implemented as separate oc '^ data ( s * ea ^ 92 ™ i0 ^P"* 6 tributaries (also 

ASICs, but instead implemented as a single ASIC. Still 20 kn 0WQ as "channels 1 ) The decapsulation elements 96 

further, the functionality of the ASICs 54, 56, 58 and 60 may ^ecysuUtc the data within each of the channels to remove 

. . , 4 , ■ / ,< ,i u j a I the data from the SONET frames and the OSI layer 2 frames, 

be implemented m software rather than ^ hardware. Mso ^ p rocC ssing elements 98 process ATM data 

the QoS Portions 70a and Tin may be distnbuted throughout ^ . q ^ ^ » m flow * similarl the £ m t processing 

the ASIC 60, or even throughout the lme card 59. ^ elements m pTQCtss ^ Ip data packetfi m (he ^ data 

Optionally, the line cards 53 have SONET multiplexers, fl 0 w. According to the illustrative embodiment, the same 

such as multiplexers 50 and 52, positioned at the line card physical elements (e.g., the receive ASIC 70 on the line card 

input ports to multiplex the incoming tributary data streams 59) process both the ATM data cells and IP data packets in 

into OC-48 data streams. In the example depicted in FIG. 4, t he input data flow. Additionally, the input processing ele- 

the SONET multiplexer 50 multiplexes four OC-12 data 3Q m ents 98 and 100 also classify the IP data packets and the 

streams 50a-50d into an OC-48 data stream. The control aa t a cells in the input data flows, according to any 

processor 64 controls operation of the line cards and 53, 55, detected ATM and IP QoS features associated with the 

57 and 59, along with the interconnect 62. packet or cell. The input processing elements 98 and 100 

An example is helpful to illustrate data flow through the also police the input data stream to detect nonconforming 

components depicted in FIG. 4. Suppose that four OC-12 35 ATM data cells and IP data packets, which exceed agreed 

data streams are multiplexed into a single OC-48 input data upon service contracts. The input processing elements can 

stream at the input port for line card 59. The receive ASIC mark or drop the policed cells. As discussed in further detail 

70 on line card 59 determines where to direct ATM cells below, the communication node 10 employs a variety of 

and/or IP packets in the input data stream. The QoS portion methods for determining which, if any, policed cells and 

70a classifies the input data stream based on any ATM or IP 40 packets to discard. 

QoS features that apply to the input data stream. The QoS Subsequent to input processing, the data passes through 

input portion 70a also prioritizes the transfer of the input the interconnect 62 to an output line card, for example line 

data stream across the interconnect 62, based on the prior card 53. The output line card includes the output processing 

classification. The interconnect 62 forwards the data stream elements 102. Based on the QoS classification of the data, 

to a destination line card, such as line card 53. The transmit 45 the output processing elements 102 perform traffic 

ASIC 71 on the line card 53 packages the data (i.e. scheduling, shaping and congestion control. Scheduling 

encapsulates) in a format that is appropriate for the desti- generally refers to selecting cells and packets among those 

nation. The QoS output portion 71a schedules and shapes eligible for transmission to send at any give time, based on 

the output of the packaged data, based on classification transmission priorities of the data and the amount of data 

information provided by the QoS input portion 70a. The 50 that has been sent for each flow, where shaping more 

QoS output portion 71a may also drop or mark the packaged typically refers to determining which data packets and cells 

data, based on the congestion status at the output ports. The are eligible for transmission at any given time so that 

data is then sent out over the output ports. Optionally, a outgoing data of each flow conforms to the service contract 

demultiplexer 50 of an access shelf 12 demultiplexes the for the data flow. According to the illustrated embodiment, 

output data from multiple tributaries into a single OC-48 55 a transmit ASIC 71, having QoS elements 71a performs the 

output data stream. output processing of both ATM data cells and IP data 

FIG. 6 is a functional block diagram 80 depicting three packets. The communication node 10 also includes encap- 

primary stages through which data transits in the digital sulation elements for encapsulating the data over a plurality 

communication node 10 of FIG. 1. More particularly, node of output channels. The communication node 10 further 

10 performs input processing 82, followed by ATM switch- 60 includes an OC-48 output multiplexer for multiplexing the 

ing and IP routing 84, followed by output processing 86. As plurality of output channels from the encapsulation elements 

described in more detail below, the input processing stage 82 104 to produce a single OC-48 output data stream 108. 

decapsulates and segments the incoming data and also According to the illustrated embodiment, the encapsulation 

locates the ATM cells and IP packets within the incoming elements 104 and the multiplexer elements 106 are included 

data stream. The input processing stage 82 also classifies and 65 in the output ASIC 71. 

polices the incoming data with regards to any associated FIG. 8 provides a conceptual block diagram depicting the 

QoS features, determines a destination handle (DH) and logical components of a QoS facility 110, according to an 
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illustrative embodiment of the invention. The components of cells outputted from the communication node 10 in accor- 

the QoS facility 110 logically divide along a control path 112 dance to the agreed upon service contract, 

and a data forwarding path 114. The logical components of jh e po ii cmg component 122 is the logical mechanism in 

the control path 112 include a call control component 116, me data f orwarding pam u 4 tna t detects if a data flow 

and a traffic control component 118. The logical components s excccds me param eters of an agreed upon service contract, 

of the data forwarding path 114 include a , data classification It a lishes this b determining if a particular packet or 

component 120, a policing component 122 and scheduling, u fa ^ > ^ mM for ^ ^tcc 

shaping, and congestion contol component 124. As d*- ^ * illustrative QoS facility 

cussed in more detail below, the illustrative QoS facility 110 * f. . ... . ... 

is physically implemented in the switching shelves 12, and ,„ U»P^nM|»toajgi«ing a leaky bucket algorithm _The 

partially controlled by a control processor (CP), which 10 buc f ^ onthm ^ * le f ^ ^ «?d«m 

resides in the control shelves 1«. al a s P eclfi < ^ he ^ ° f bu ' ket Tf^V 

"r „ , _ . ., maximum burst size for the particular flow. If the flow 

The call control component 116 provides a mechanism by <. overflows » me bucket ^ QoS facilit 110 detects me 

which the communication node 10 of FIG. 1 accepts or overflowin kels or ^ M bei nonconformmg and 
rejects requests for various levels of QoS from external 15 polices ^^epending on the service contract, the poUc- 
sources or from external delations attempting to reserve ^p^nt U2 either marks or discards any non- 
bandwidth for data flows that the external destinations co 6 nform £ cel]s or ackets . According , 0 me mustrlled 
expeel ito receive. Accordmg tc .the lUustra^e ernbodime^ 6mbodimcnt , occurs in mr6C plac6s; ^ ajm 
the call coQtrol component 116 employs ATM UNI 4.0 and , ^ ^ ue m of FJG 
RSVP to enable external networks to reserve bandwidth. M ^ p £ discj ^ ed ^ ^ ^ ^ 
The call control component 116 interacts with the traffic " 

control component 118 to configure the data forwarding path ^ A™ looku P of ™. 21 performs a standard ATM 

114 to provide an agreed upon QoS. Agreements between Usa 8« Parameter Control (UPC) algorithm with two leaky 

the communication node 10 and external networks for buckets for the peak and sustained cell rates (PCR and SCR), 

particular QoS features are referred to hereinafter as service „ ™ e Q° s fac ility ™ ™*» nonconformmg cells using the 

contracts. The call control component 116 also determines * Cell Loss Pnonty (CLP) bit and polices the cells based on 

the available bandwidth of the communication node 10, and ^ properties of the VC According to one preferred 

rejects or accepts a request for a service contract, based on embodiment, the QoS facility 110 does not discard cells 

whether sufficient bandwidth exists to fulfill the QoS unless «» °^P M 1 ueues of me communication node 10 

requirements of the service contract. 30 become congested. In this case, the CLP bit proves an 

~, , a- . , * -lie ■ *u i • l „ u indication of which cells are preferable to discard. The 

The traffic control component 118 is the logical mecha- *; . , 4 , , . 

. a *u * , a „*„ ta • „ A „ to output queue manager, discussed in more detail below and 

msm that configures the appropriate state in the components £ & ' . 

of the data forwarding pail 1 114. The traffic control compo- ™& ° 3 *> can dro P the CLP bit set 

nent 118 provides configuration parameters to the classifi- * the out P ut ^ eues become 

cation 120, policing 122 and scheduling, shaping and con- 35 n * IP looku P of na 26 determines whether IP input 

gestion control component 124. Additionally, the traffic data packets are within a differentiated services of integrated 

control component 118 interprets parameters from the call services P rofile * ™ e IP looku P sets bits * the DH t0 mdicate 

control components 116 used to configure the service whether an IP data packet is "in" or W of profile, 

contract, and translates those parameters into a format According to the illustrative embodiment, the IP lookup, 

required by the logical components along data forwarding 40 optionally, discards nonconforming packets. The queue 

path 114 manager 620 of FIG, 33 uses the results of the IP lookup. In 

Hie classification component 120 is the logical mecha- ^ event mat te queues become congested, the 

nism in the data forwarding path 114 that processes the IP <f eue ( man , a 6 e f r 15 m ° re likelv lo ***** P ackets marked as 

data packets and the ATM data cells in the input data flow, bein S 0111 °fp ronle - 

and determines which packets and cells in the input data 45 The queue manager 620 of FIG. 31 also performs policing 

flow require a particular QoS. In response to determining for traffic that is shaped. The queue manager may discard IP 

that a cell or packet requires a particular QoS, the classifi- data packets and ATM data cells that are scheduled too far 

cation component 120 classifies the cell or packet based on away in the future from the current time, 

the required QoS. To achieve bandwidth enhancements, the ATM Adaption Layer 5 (AAL5) provides the mechanism 

illustrative communication node examines input data one 50 for taking an ATM frame and segmenting it into a series of 

time. Accordingly, lookup engines provide the QoS classi- ATM cells. The ATM cells are reassembled at the ATM cell 

fication for the input data. For ATM, this is implicit in the destination. According to the illustrative embodiment, if 

Virtual Path Indicator/Virtual Channel Indicator (VPI/VCI) some of the cells that make up an AAL5 frame exceed a 

for the VC. For IP, the classification component 120 utilizes traffic contract and require discarding, the QoS facility 110 

a combination of one or more of the destination address, the 55 avoids sending at least a portion of the frame containing the 

source address, the IP protocol number, and the source and policed cell. The illustrative QoS facility 110 employs two 

destination port. The result of the lookup is the Destination independent methods for avoiding sending the portion of the 

Handle (DH), which specifies the destination descriptor. The policed cell. One method is Partial Packet Discard (PPD). 

destination descriptor is unique to each data flow. The DH The other is Early Packet Discard (EPD). 

specifies, among other parameters, the output port and eo According to PPD, if the policing occurs in the middle of 

output queue (discussed below with respect to FIGS. an AAL5 frame, then the QoS facility 110 drops all but the 

33-34). The DH also specifies whether the data conforms to last cell of the remaining cells in the frame. The QoS facility 

the service contract of its source. 110 allows the last cell to remain because it contains an end 

The scheduling, shaping and congestion control compo- of frame indicator. When the checksum for the frame fails, 

nent 124 is the mechanism that sorts out cells and packets 65 the destination discards the frame. In this way the QoS 

from multiple data flows and ensures that data flows requir- facility 110 reduces the number of cells that pass through the 

ing particular QoS features have their associated packets or communication node 10 during periods of congestion. 
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According to EPD, if the traffic over the interconnect 62 bandwidth of traffic from or to a particular set of customers, 

of FIG. 4 is congested, which can be detected when the addresses, or autonomous systems based on a time schedule, 

to-interconnect queue size exceeds some predetermined The QoS facility 110 configures the classifier 120, the 

threshold, the QoS facility 110 discards any newly arrived policing module 122 and the scheduler 124, according to 

nonconforming AAL5 frames, until the queue size drops 5 which embodiment is employed. 

below the threshold. This preserves AAL5 frames as much Differentiated Services is concerned with providing ser- 

as possible, without allowing partial frames to propagate v { cc differentiation for "best effort" services through simple, 

through the communication node 10. scalable mechanisms. The QoS facility 110 applies this 

According to another feature of the invention, PPD and approach to adaptive applications that require a low, but not 

EPD are also used by the illustrative QoS facility 102 to 10 necessarily bounded, delay. Differential Services allows the 

control traffic congestion at the output ports. In addition, service provider to offer a service contract that guarantees a 

QoS facility 102 may also use Random Early Detect (RED) minimum bandwidth or rate through the communication 

to randomly discard data when the average lengths of the node 10, but allows the customer to use more bandwidth 

output queues exceed certain thresholds. when the node 10 is not congested. The QoS facility 110 

According to a further feature of the invention, the 15 implements Differential Services by modifying the ToS/ 

illustrative QoS facility 110 determines the source address of Precedence bits in the Ipv4 header. These bits are used to 

data sources that habitually send nonconforming data indicate whether the customer's packets are to receive a 

streams, and penalize those sources. One method employed particular QoS, and whether the traffic is "in" or "out" of the 

by the QoS facility 110 for penalizing offending sources to P rofile for toe customer. If the traffic is "out" of profile, the 

assign those sources a lower priority for a period of time. 20 QoS facility 110 has the option to discard the traffic if 

Another method is to mark data from offending sources as congestion is encountered. The goal is to avoid discarding 

non-conforming so that they will be more likely to be packets that are "in" profile, while allowing traffic that is 

discarded in case of congestion. " out " of P rofile if resources are available. 

A QoS facility 110, according to the illustrative embodi- Differential Services allows service providers to design 

ment of the invention, supports ATM QoS service categories service profiles that meet the customers bandwidth and delay 

defined by the ATM Forum. The ATM Forum has defined needs, without installing complicated protocol mechanisms 

five service categories for ATM. As previously mentioned, to implement them. The edge routers test the packets based 

these service categories include CBR, rtVBR, nrtVBR, UBR on the customer profile while the core routers only test 

and ABR. When the communication node 10 sets up a VC, whether a given packet is "in" or "out" of profile. By 

it specifies one of these service categories. With each service providing service profiles, the service providers can make 

category comes and associated set of QoS parameters that competitive packages that allow customers to pay for better 

are specific to the category. Below is a brief description of service if they require it. 

each of the ATM service categories. The Integrated Services architecture, as outlined in RFC 

CBR is characterized by a peak cell rate that is continu- 35 1633, provides a range of services, better than traditional 

ously available during the connection lifetime. CBR requires "best effort" services, to applications. One notable applica- 

that delay variations be kept to a requested value if traffic tion for Integrate Services architecture is for real-time 

over the connection does not exceed the peak rate. Cells applications using audio and video. These applications 

delayed beyond a maximum specified value are useless to require an end-to-end QoS for one or more application data 

the application. rtVBR is characterized by peak and sus- 40 flows, thus requiring a protocol to signal to the communi- 

tained cell rates along with a maximum burst size. A user cation node 10 regarding the needs of the application, 

may send traffic over the connection at the peak cell rate of including what resources are needed and how to identify the 

an amount not exceeding the maximum burst size, and the application data flows. The protocol that signals this infor- 

average traffic rate must not exceed the sustained cell rate. mation is the ReSerVation Protocol (RSVP). Integrated 

Delay variations are expected to be kept to a requested 45 Services combines RSVP with the Integrated Services 

value. Delays beyond a specified maximum Cell Transfer (IntServ) Models for the Controlled Load and Guaranteed 

Delay (maxCTD) are useless to the application. nrtVBR is Services to create end-to-end per flow QoS. Another piece of 

characterized by the same peak, sustained and burst size the IP QoS deals with the mapping of the IntServ models to 

parameters as the rtVBR category. However, cell delay specific media (ISSLL). 

variations are not a significant factor. UBR is a best effort 50 RSVP enables an end system to request that the commu- 

service that is characterized only by an optional peak cell nication node 10 provide special treatment for a flow or set 

rate. No delay or bandwidth guarantees are made. ABR is a of flows. It provides a filter specification indicating what 

service that provides feedback regarding network conges- packets should receive the QoS that is used by the classifier 

tion to allow sources to adjust their transmission rates to 120, as well as the input data flow specification. The input 

reduce cell loss. 5S data flow specification indicates what QoS level the classi- 

The QoS facility 110 also supports three different types of fied packets should receive. The flow specification is used by 
IP QoS. Those are Configured/Provisioned QoS, Differential the traffic control module 118 to program the policing 122 
Services, and Integrated Services/RSVP. Though, these are and scheduling modules 124. RSVP operates in a "receiver- 
well known in the art, a brief description of each is provided based" mode where the data receivers are the ones that 
below. 60 actually make the reservation through the network. This 

Configured/Provisioned QoS is a scheme whereby the allows RSVP to scale to large multicast distributions, with 

administrator configures an IP router to statically provide a reservation merging at split points in the multicast distribu- 

QoS to certain traffic classes passing through the router. uon trcc * 

According to one embodiment, the QoS facility 110 accom- Another aspect of IP QoS is the INTSERV models devel- 

plishes this by prioritizing packets to a particular TCP port 65 oped by the INTSERV working group. There are two models 

(i.e. DLSW or Telnet) over other traffic. According to a currently standardized. These are Controlled Load Service 

further embodiment, the QoS facility 110 allocates the and Guaranteed Service. These models provide the param- 
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eters for the RS VP flow specification and specify the behav- necessary. For simple data flows, the DH specifies the output 

ior of the policing 122 and scheduling 124 modules. port and the output queue for the data, allowing the sched- 

According to Controlled Load Service, the QoS facility uling mechanism 144 to direct the data to the proper output 

110 provides a service level that corresponds to an unloaded queue 146. 

network. Delay bounds are not explicitly stated but are s The receive ASIC forwards traffic to the interconnect 62 

expected to be consistently lower than "best effort" traffic inlQ lhree queues ^ receive as IC 70 determines in which 

with low variation. Applications using this service are c tQ lacc {hc d bascd on mc destination queue 

expected to be adaptive to variations in delay. idcn^d in the DH. This allows high priority data, such as 

According to Guaranteed Service, the communication ^ CBR ^ tQ bc rioriti2ed ovcr lowcr riority traf6c 

node 10 provides a service level that corresponds to a fluid 30 as i{ entefS ^ interconnect mile m prioritization is at a 

model, wherein the commutation node 10 behaves as a q grmu{uii y than the output queuing, the higher speed 

pipe sized precisely to the needs of the application, and the r ? 4 ' „ r , T, . <? ^ . r 

delays are bounded L The communication node 10 advertises of * e 62 prevents this from causing any 

its transit delay in RSVP PATH messages so a receiver can P rable * s meeUD 8 QoS guarantees. 

determine if the node 10 has the available bandwidth to A back pressure mechanism prevents low priority traffic 
support the bounds required. The communication node 10 15 from swamping a line card 130-138 by allowing a line card 
can support a wide variety of multimedia applications using 130-138 to request, through the interconnect 62, that 
these two service models. another line card 130-138 "back off" on data transmission of 
As mentioned above, the third piece of IP QoS is directed data having a particular priority. This mechanism prevents a 
toward the mapping of the IntServ models to specific media. li Qe card having an abundance of effort traffic from swamp- 
The Integrated Services over Specific Link Layers (ISSLL) m an output line card having data flows that require a 
working group is responsible for this part of the puzzle. higher level of QoS. The interconnect 62 may also generate 
Because IP is a layer 3 protocol and relies on a variety of back pressure signals to avoid congestion within the inter- 
layer 2 media, a mapping is required between the layer 3, connect. 

IntServ models, and the native QoS abilities of the layer 2 ^ The operation of the classification and policing elements 

media. For some media, this means that real QoS is not 142, the shaping, scheduling and congestion control ele- 

really possible without some compromises. The illustrative ments 144, the output queues 146, and the interconnect 

communication node 10 employs standard mapping of the prioritization elements 148 are discussed in further detail 

IntServ models to ATM, Frame Relay and Gigabit Ethernet. below with respect to the input ASIC 70, shown in more 

FIG. 9 is a schematic block diagram of a portion 126 of 30 detail in FIG. 13. 

the switching shelf 12 of FIG. 2. FIG. 9 illustrates some of As discussed above, the illustrative QoS facility 110 

the logical components of the of the QoS facility 110 of FIG. performs classification 120 and policing 122 during input 

8. The depicted portion 126 includes five line card modules processing. FIG. 10 shows a flowchart 160 that illustrates 

130-138 and interconnect 62. A control processor 64 is also the steps that are performed during input processing in the 

depicted. A control network 141 electrically connects the 35 illustrative embodiment. Initially, according to step 162, the 

line cards 130-138. Each line card 130-138 includes clas- incoming data is demultiplexed into the respective SONET/ 

sification and policing elements 142, shaping, scheduling SDH tributaries. Next, as shown at 164, the receive ASIC 70 

and congestion control elements 144, a queuing structure decapsulates the input data stream. Next, the receive ASIC 

146, prioritization elements 148, a line card processor (LCP) 70 determines whether the input data is an ATM cell (step 

72, a receive ASIC 70 and a transmit ASIC 71. 40 166 in FIG. 10) or an IP packet (step 170 in FIG. 10). If the 

The line cards receive data via input ports 150, classify input data is an ATM cell, then the receive ASIC 70 performs 
and police 142 the data and send it to the interconnect 62. ATM input processing (step 168 in FIG. 10). Alternatively, 
The prioritization elements 148 prioritize the data over the for an IP packet, the receive ASIC 70 performs IP input 
Interconnect to ensure that time critical data is delivered on processing (step 172 in FIG. 10). IP and ATM input pro- 
time. The shaping, scheduling and congestion control ele- 45 cessing are discussed below in further detail, 
ments 144 shape, schedule, and flow control data coming FIG. 11 depicts a more detailed flow diagram 180 of input 
from the interconnect 62 according to the QoS of the data processing. The SONET demultiplexers 94 demultiplex the 
flow and congestion status, and places the data in an OC-48 data stream 92. The resulting data in the respective 
appropriate priority output queue 146 for transmission. As tributaries may be in any of a number of different formats, 
discussed in more detail with respect to FIG, 33, the CP 128 50 As shown at 96, the receive ASIC 70 decapsulates the 
controls the classification, policing, shaping, scheduling and demultiplexed data stream (step 162 in FIG. 10) to gain 
congestion control of the data flows by populating tables access to the ATM cells or IP packets carried in the input 
with the appropriate information for the transmit ASIC 71 to data stream. The receive ASIC 70 is adapted for decapsu- 
make decisions. This population occurs through LCP 72. lating a number of different types of OSI layer 2 encapsu- 

In a time saving feature and as discussed below with ss lations. The decapsulation step 162 of FIG. 10 may also 

respect to FIG. 13, the receive ASIC 70 performs classifi- include the deframing of SONET frames, 

cation of incoming data in conjunction with the routing FIG. 12 depicts the format of a SONET frame 200. The 

lookups. For ATM and Frame Relay data, this is simply a SONET frame 200 includes 9-rows, each row containing 90 

matter of looking up the incoming VPI/VCI and DLCI, Octets (i.e. 90 8-bit bytes). The payload for the SONET 

respectively. For IP data, the lookup can include matching 60 frame 200 is contained in the synchronous payload envelope 

the source address, IP protocol type, and TCP or UDP port (SPE) 202. The SPE 202 contains 9-bytes that are dedicated 

numbers, as well as the destination address. By performing to path overhead (OH) 208. The SONET frame 200 also 

this kind of long lookup at the input, the illustrative QoS contains section OH 204 and line OH 206. The section OH 

facility 110 saves the time required to look up the same 204 and line OH 206 are part of the SONET transport 

information on the output. 65 overhead. In this context, "overhead" refers to header infor- 

The result of the routing lookup is a DH. The DH provides mation that is provided for use by various layers of the 

a mechanism for looking up further parameters for QoS, if computer network. 
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FIG. 13 depicts the logical components of the receive 
ASIC 70 in more detail As skilled artisans will appreciate, 
the divisions between the depicted components of the 
receive ASIC 70 arc only illustrative in nature, and can be 
alternatively drawn, without impacting the scope of the 
invention. The receive ASIC 70 includes a SONET deframer 
210 that receives the input data. The SONET deframer 210 
removes the contents of the SPE 202 from the SONET frame 
200. The resulting payload may contain additional 
encapsulations, as will be described in more detail below. 

FIG. 14, by way of example, shows how the payload of 
the SONET frame 200 can contain one or more DS-3 PLCP 
(Physical Layer Convergence Protocol) frames 260. Such a 
frame 260 holds a payload that is used in mapping ATM cells 
onto DS-3 facilities- The frame 260 includes PLCP framing 
Octets 262 to identify the framing pattern that is utilized. 
The path overhead indicator (POI) 264 indexes the adjacent 
path overhead (POH) Octets 266 and identifies the encoding 
for the POH octet. The payload 268 includes the data content 
for the frame 260. The frame 260 may also include trailer 20 
nibbles (i.e. 4-bits) 220. 

FIG. 15 depicts an alternative embodiment in which the 
data is encapsulated in a point-to-point protocol (???) frame 
280. PPP is an OSI layer 2 protocol that is built on top of a 
restrictive subset of the standard high level data link control 
(HDLC) protocol. Each PPP frame 280 includes an address 
282 and a control field 284 for holding flow control infor- 
mation. The PPP frame 280 contains a HDLC information 
section 286 that is 1502-octets long and contains a PPP 
payload. The CRC field 288 identifies the cyclic redundancy 
check that is used for the frame. The PPP frame 280 also 
includes frame delimiter flags 281 and 289. 

FIG. 16 shows another alternative embodiment in which 
the data is encapsulated in a Frame Relay (FR) frame 290. 
FR is an OSI layer 2 protocol. Each FR frame 290 includes 
a byte of flag information 292 and an address field 294. The 
FR frame 290 also contains an information field 296 that 
holds a payload and a frame check sequence octet 298. The 
octet 298 that supplies information used to check whether 
the frame is properly received. Lastly, the FR frame 290 has 
a flag octet 300 at the end of it. 

As shown in step 162 of FIG. 10, decapsulation includes 
removing the ATM data cells or the IP data packets from the 
OSI layer 2 frames (such as FR frame 290 or PPP frame 280) 
or from the PLCP frame 260. The receive ASIC 70 maintains 
interface information regarding the input port through which 
input data is received. The interface information includes a 
separate context for each input data tributary/stream and the 
context identifies the nature of the tributary. Hence, as 
shown in FIG. 13, for pure ATM, the output from the 
SONET deframer 210 is passed to the PCLP deframer 212. 
If data contains PPP frames or FR frames (as indicated by 
the context) the data is sent to the PPP/FR deframer 214. 

FIG. 18 depicts an illustrative ATM data cell 310. Each 
ATM data cell 310 is 53 bytes long with 48 bytes of payload 
314 and 5 bytes of header 312. The header portion 312 
includes generic flow control 316; a Virtual Path Indicator 
(VPI) 318-320; a Virtual Channel Indicator (VCI) 322-326; 
a payload type 328; a cell loss priority 330; and a header 
error check (HEC) 332. The generic flow control field 316 
can be used to provide standardized flow control at a 
customers site. The VPI 318-320 identifies a virtual path 
(VP) for the ATM cell 310. The VCI 322-326 identifies the 
VC for the cell. ATM cells use VCIs and VPIs to specify 
treatment of a cell. A VC is a connection between two 
communicating ATM entities. A VP is a group of VCs that 



is carried between two points. VPs provide a convenient 
technique for bundling traffic that is heading for same 
destination. In some instances, the node 10 need only check 
for a VPI to relay traffic rather than checking a more 
5 complete address. 

The payload type 328 includes a 3 -bit field that indicates 
whether the cell 310 contains user information or contains 
associated layer management information. The cell loss 
priority bit 330 allows the specification of explicit loss 
10 priority for the cell The header error control field 332 is 
used by the physical layer of the node 10 for detecting bit 
errors in the cell header 312. 

Each PLCP frame 260 of FIG. 14 may contain up to 
12-ATM cells 310. After the PLCP frame 260 is deframed, 
the ATM cells are located within the payload. As shown in 
FIG. 13, an ATM header error control (HEC) delineator 216 
locates the ATM cells 310 within the PLCP payload 268. 
Once the ATM cell 310 has been located and the header 
located for the ATM cell, the receive ASIC 70 can perform 
the ATM input processing of step 168 in FIG. 10. 

FIG. 19 is a flow chart 350 illustrating the input process- 
ing performed by the receive ASIC 70 of FIG. 13. With 
reference to FIGS. 13 and 19, the HEC delimiter 216 of FIG. 
13 forwards the ATM cell header 312, along with input port 
information, to the ATM lookup engine 220. The HEC 216 
sends the remaining 48-bytes of the ATM cell 310 to the 
receive FIFO 222. 

As mentioned above, the HEC delimiter 216 sends the 
ATM cell header 312 to the ATM lookup engine 220 (step 
352 in FIG. 19). The HEC delimiter 216 sends the payload 
314 to the receive FIFO 222 (step 354 in FIG. 19). The ATM 
lookup engine 220 uses an AIM table 224 to perform a 
lookup to determine where to direct the ATM cell 310 (step 
358 in FIG. 19). Additionally, the ATM lookup engine 220 
plays a role in both the ATM policing (see 122a in FIG. 11) 
and the ATM lookup function (see 182 in FIG. 11). As 
illustrated, the ATM cell header 312 that is sent to the ATM 
lookup engine 220 does not include the HEC field 332. Also, 
according to the illustrative embodiment, the ATM lookup 
engine 220 performs a lookup in parallel with the payload 
314 being stored in the receive FIFO 222. 

The discussion below focuses first on the performance of 
the ATM lookup engine 220, and then describes policing 
performed by the ATM lookup engine 220. As previously 
discussed, the policing determines whether incoming data is 
in conformance with agreed upon service contracts. It 
accomplishes this by measuring traffic rates, and comparing 
actual rates with the contracted for rates. 

As shown in FIG. 21, an illustrative incoming ATM cell 
310 goes through a three stage lookup 400. The first stage 
involves accessing the port lookup table (PLUT) 402. The 
PLUT 402 contains 49-entries, where 48-entries are pro- 
vided for the 48 -different input ports that are possible and a 
49 th entry corresponds to the line card processor (LCP) 72. 
Each entry in the PLUT 402 points to an entry in the VP 
lookup table (VPLUT) 404, which constitutes the second 
stage of the lookup. Each entry in the VPLUT 404 is 
associated with a particular VP. Hence, an entry in the 
VPLUT 404 points to the VP associated with the input port 
context for the entry. Each VPLUT entry 404 points to a VC 
lookup table (VCLVT) 406 which holds information for a 
particular VC. Each entry contains 128 bytes of data. The 
data identifies the VC to which the ATM data cell 310 is 
routed or switched, or indicates that the VC terminates on 
the LCP. 

To reduce propagation delays through the illustrative 
communication node 10, incoming data is examined only 
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once. The resulting DH provides the key for finding any 
further information for a data flow. The DH specifies the 
Destination Description which is unique to the data flow. 
Additionally, the DH specifies the output port and output 
queue to which the data should be directed. The DH also 
includes indicators regarding conformance to policing and 
regarding QoS contracts, as well as an indicator that further 
QoS lookups are necessary to schedule data flow. 

As previously mentioned, to stop data flows from inter- 
fering with each other, it is sometimes necessary to police 
data flows to make sure they are not exceeding their con- 
tracts. Policing is accomplished using a "leaky bucket" 
algorithm that allows a bit bucket to drain at a specific rate 
while the depth of the bucket matches the maximum burst 
size for the flow. If the flow "overflows" the bucket, the 
packets are considered "non-conformant" and can be 
policed. As previously mentioned, policing is performed in 
three places in the COMMUNICATION NODE 10; the 
ATM lookup (SEE FIG. 21), the IP lookup (see FIG. 26), and 
the Queue Manager (see FIG. 13). 

The ATM look up engine 220 employs policers for 
monitoring peak cell rate (PCR) and sustained cell rate 
(SCR), according to the traffic service contract for the VC or 
VP. Each policer implements the generic cell rate algorithm 
(GCRA), which is defined in the UNI 4.0 specification. The 
PCR leaky bucket algorithm monitors the maximum cell rate 
within the tolerance permitted by the cell delay variation 
toleration (CDVT). The SCR leaky bucket algorithm moni- 
tors the average cell arrival rate over a period of time within 
the burst size permitted by the maximum burst size (MBS) 
and CDVT. SCR applies to VBR and UBR connections. 
Traffic contracts are defined in accordance with the ATM 
Forum Traffic Management 4.0 specification. 

Non-conformant cells are dropped or marked using the 
CLP (Cell Loss Priority) bit and are policed based on the 
properties of the VC. If marked, the cell is not discarded 
unless there is congestion in the output queues. The CLP 
provides an indication as to which cells are the best to be 
dropped. If the output queues (shown at 622 in FIG. 33) 
experience congestion, the output queue manager (shown at 
620 in FIG. 33) discards cells with the CLP bit set. 

The lookup engine 220 sends the results of the ATM 
lookup (i.e. the DH) to the receive FIFO 222 (step 360 in 
FIG. 18). Optionally, the ATM lookup engine 220 can elect 
discard a cell as part of policing (step 362 in FIG. 18). The 
nonconforming cells are then discarded (step 368 in FIG. 
18). If the cell is not to be discarded, the ATM lookup engine 
220 requests a ticket from the ticket master 232 (step 270 in 
FIG. 18). The ticket is a pointer to a location within a receive 
data parking lot 230. The receive data parking lot 230 is a 
place for storing data while processing is being performed. 
The ticket may be redeemed to extract the data from the 
location identified by the ticket. In response to a request, the 
ticket master 232 issues a ticket and sends the ticket to the 
ATM lookup engine 220 (step 372 in FIG. 18). The receive 
FIFO 222 then transfers the ATM payload 314 to the receive 
data parking lot 230. The receive data parking lot stores the 
data at the location identified by the issued ticket (step 374 
in FIG. 18). The receive data parking lot 230 forwards the 
48-byte payload portion 314 of the ATM cell 310, along with 
the ticket and the DH, to the canonical decapsulation module 
252. The canonical decapsulation module 252 includes a 
decapsulation table 184 that determines how first to decap- 
sulate the data, then to encapsulate the data into the internal 
cells (i.e. canonical format) used by the communication 
node 10. For raw ATM cells, the payload 314 is combined 
with the header information and the DH to create the internal 
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data cell. In particular, the canonical decapsulation module 
252 constructs an internal cell with the format depicted at 
420 in FIG. 20. The internal cell 420 includes a data portion 
426 as well as the DH 424. The internal cell 420 also 

s includes an interconnect header portion 422. The intercon- 
nect head portion 422 provides header information that is 
used by the interconnect 62. 

In step 170 of FIG. 10, the receive ASIC 70 may deter- 
mine that the incoming data is not an ATM cell, but rather 

10 is an IP packet or an ATM cell containing an IP packet or part 
of an IP packet. In such a case, the receive ASIC 70 performs 
IP input processing (step 172 in FIG. 10). The IP packet may 
be encapsulated in a PPP frame 280 of FIG . 15 or a FR frame 
290 of FIG. 16. As was mentioned above, the deframer 214 
deframes the PPP frames 280 and the FR frames 290. The IP 

35 packet also may be encapsulated in an AAL5 (ATM adap- 
tation layer 5) frame. In other words, the IP packet may be 
transmitted over ATM. 

FIG. 17 depicts the format of an AAL5 interface data unit 
(IDU) 430. The IDU 430 includes a payload 432, as well as 

20 a trailer 434. The IDU 430 may be of variable length. The 
trailer 434 contains the User-to-User (UU) field 436, which 
holds data that is to be transferred transparently between 
users. A Common Part Indicator (CPI) field 438 aligns the 
trailer in the total bit stream. The length field 440 indicates 

25 the length of the total IDU payload 432. A cyclic redundancy 
check (CRC) field 442 is used for error detection correction 
in the trailer only. The entire set of data contained in the IDU 
430 is segmented into 48-octet payloads prepended with a 
5-octet header to form 53 -octet ATM cells. 

30 FIG. 22 is a flowchart 430 that illustrates the steps 
performed by the receive ASIC 70 during input processing 
for IP packets. The AAL5 segmenter 218 divides the IP 
packet into pseudo-ATM cells (step 400 in FIG. 20). In the 
case where the input data is in packet over ATM format, the 

35 IP packet may be held in one or more AIM cells. The AAL5 
segmenter 218 sends the header information 312 from each 
of the pseudo ATM cells to the ATM lookup engine 220 (step 
434 in FIG. 22). The ATM lookup engine 220 recognizes the 
cell as containing IP packet data and places the header 

40 information 312 for the cells in the pending cells queue 236 
(step 436 in FIG. 22). The pending cells queue 236 accu- 
mulates the header information for the cells that constitute a 
single packet. In this way, the receive ASIC 70 ensures that 
all of the cells for an IP packet have been processed before 

45 transmission of internal cells for the IP packet over the 
interconnect 62. 

In order to understand how processing proceeds, it is 
helpful to consider the case where a PPP frame 280 of FIG. 
15 contains an IP packet. In such an instance, the receive 

50 ASIC 70 shreds the IP packet in the PPP frame 280 into 
pseudo-ATM cells and sends the headers 312 to the ATM 
lookup engine 220 and the 48-data bytes of the payload 314 
to the receive FIFO 222. The PPP VPs and/or VCs are 
aliased into ATM cells so that ATM lookup engine 220 is 

55 able to process the pseudo-ATM cells. Specifically, traffic 
coming over a PPP context has a VPI with a preconfigured 
value of zero. This value is inserted into the PPP frame 
before sending the packet to the AAL5 segmenter 218. The 
VPI value of zero for the PPP context is set up as a switched 

60 VP circuit that is routed. For FR frames 290, the VPI is set 
to the incoming DLCI value plus one. When processing 
incoming header data, the ATM lookup engine 220 returns 
either a DH or a placeholder DH. The placeholder DHs are 
an indication that the incoming header is for an IP packet. 

65 The presence of the placeholder DH output causes the ATM 
lookup engine 220 to put the header information in the 
pending cells queue 236. 
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The ATM lookup engine 220 determines whether the 
input data cell is for the first cell of an IP packet (step 438 
in FIG. 22). If the ATM lookup engine 220 determines that 
the cell is first cell for an IP packet, it sends the IP header 
information to the first cell decapsulation module 240 (step 
440 in FIG. 22). Additionally, the receive FIFO 222 sends 
the 48 bytes of payload data for the cell to the first cell 
decapsulation module 240, as well (step 442 in FIG. 22). The 
first cell decapsulation module 240 decapsulates the infor- 
mation contained in the header to send appropriate infor- 
mation to the IP lookup module 244 (step 444 in FIG. 22). 
The first cell decapsulation module 240 uses a decapsulation 
table 241 to identify how to decapsulate the cell. The IP 
lookup module 244 performs both forwarding lookup 184 
(see 184 in FIG. 11) and policing (see U2b in FIG. 11) for 
IP packets. 

As the lookup is proceeding, the interconnect 62 requests 
a ticket from the ticket master 232 to obtain data to be sent 
over the interconnect 62 (step 446 in FIG. 22), and in 
response, the ticket master 232 issues a ticket (step 448 in 
FIG. 22). The receive FIFO 222 transfers 48-bytes of 
payload data to the receive data parking lot 230 (step 450 in 
FIG. 20) for storage at the location identified by the issued 
ticket. The IP lookup module 244 returns a DH that identifies 
the destination line card for the internal cell that will be 
forwarded over the interconnect 62 (step 452 in FIG. 22). 
The receive ASIC 70 forwards an interconnect header, the 
DH and the 48-bits of payload data over the interconnect 62 
(step 454 in FIG. 22) in the internal cell format 420. Prior to 
transmission, the receive ASIC 70 buffers these fields in the 
receive input queue 286. 

FIG. 23 depicts the format of the IP header data 460 that 
is used by the IP lookup module 244. All of the fields in the 
header data 460, other than fields 486 and 488, are copied 
from the IP header of the associated IP packet. Fields 486 
and 488 are copied from a transport header. The header data 
460 includes a version field 462, which holds information 
regarding the version of the IP protocol being used. For 
version 4 IP packets, this field 462 holds a value of 4. The 
Internet Header Length (IHL) field 464 identifies the length 
of the header from the IP packet in multiples of 4-octets. The 
TOS field 466 holds a value that identifies a particular 
handling or treatment for the packet. The total length field 
468 holds information regarding the total length of the 
packet. The identification field 470 provides an identifica- 
tion value for the packet. If the packet is later fragmented, 
the identification value associates the fragments with the 
original packet. 

The header data 460 includes flags 472, including a DF 
flag and a MF flag. The DF ("don't fragment") flag indicates 
whether a datagram that is carried at least in part by the 
packet is allowed to be fragmented, should it be necessary. 
The MF ("more fragment") flag identifies whether there are 
more fragments or whether the packet holds the last frag- 
ment of the datagram. The fragment offset field 474 holds an 
offset value that identifies the offset in which the fragment 
belongs to the reassembled packet. The time to live field 476 
identifies the number of hops a packet may have before the 
packet is discarded. The protocol field 478 holds a value that 
allows the network layer of the destination end node to know 
to which transport protocol running at the destination end 
node the packet should be directed. A header checksum field 
480 is provided. Asource address field 482 and a destination 
address field 484 are also provided. The source address field 
stores a source address of the node from which the packet 
originated. The destination address field stores a destination 
address for the node to which the packet is to be forwarded. 
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The source port field 486 identifies a source port and 
destination port field 488 identifies a destination port for the 
packet. 

The IP lookup module 244 uses a number of tables (see 
Route Table 246 in FIG. 13) and other structures in per- 
forming IP lookup. FIG. 24 depicts a number of the more 
prominent tables and structures 500 that the IP lookup 
module 244 utilizes. An interface (IF) structure 502 is 
provided to identify each interface (i.e. context) from which 
data is received. The interface structure contains an initial 
lookup element that is utilized when forwarding lookup is to 
be initiated. This initial lookup element is an array lookup 
element that contains an instruction to be executed at the 
beginning of forwarding lookup for an IP packet. 

The IP lookup module 244 uses lookup arrays 504 con- 
taining lookup elements. The IP lookup module 244 may 
also use a SANET 506 or DANET 508. The SANET 506 is 
a data structure that provides a number of structures for 
respective source addresses that are being exploited for QoS 
processing and Type of Service (ToS) processing. The 
DANET 508 holds DANET structures that contain informa- 
tion regarding destination addresses that identifies the next 
hop for IP packets, 

FIG. 25 shows the format of an illustrative DANET 
structure 508. The DANET structure 508 includes a DH and 
a pointer to a rotor or a pointer to a ToS array in field 510. 
A rotor is a data structure that contains a set of DHs. A rotor 
may be used to aggregate multiple lower speed links into a 
virtual higher speed link. A ToS array is also an array of 
handles but it is indexed by a ToS parameter value. The ToS 
array enables the DH to vary with ToS. The DANET 
structure 508 also includes counters 512 for tracking statis- 
tical data, as well as other data. The DANET structure 508 
further includes a data field 514. 

FIG. 26 provides a flow chart 520 illustrating the steps 
that the receive ASIC 70 performs during an IP lookup for 
a unicast IP packet. The IP lookup determines how to send 
the IP packet to the next hop toward the destination (i.e. 
ultimately, it determines what output port to use). The IP 
lookup module 244 of FIG. 13 detects the interface on which 
the IP packet arrived. The interface structure for the asso- 
ciated interface is accessed and the IP lookup module 244 
processes the initial lookup element contained in the inter- 
face structure (step 522 in FIG. 26). As shown in FIG. 27, 
the interface element contains a lookup element 550. The 
lookup element 550 contains an array address 554 and an 
opcode for array lookup 558. The lookup element 550 also 
contains a header nibble select 556 that identifies what 4 bit 
nibble within the header may be utilized to generate an index 
to an array lookup element 550 in the lookup array 564. The 
array address combined with the nibble that is selected by 
the header nibble select 556 is used to access a lookup 
element 566 in the lookup array 564. The bits 562 contained 
within the header of the IP packet 560 are combined to 
produce an index for accessing the lookup element 566. 

The route table 246 of FIG. 13 contains multiple lookup 
tables. In particular, a tree of lookup arrays is provided. The 
first level of the tree is a single lookup array that is indexed 
by the first two bytes of the destination IP address for an IP 
packet. The second level of the tree contains lookup arrays 
indexed by the third bytes of the destination IP address. The 
third level of the tree contains lookup arrays that are index 
by the final byte of the destination IP address. By using this 
tree structure, the illustrative embodiment is able to decrease 
the number of memory access required, and to increase the 
speed with which IP lookup occurs. 
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After the instruction has been accessed in the interface 
structure (see step 522 in FIG. 26), an entry is accessed in 
the first lookup array and processed (step 524 in FIG. 26). 
The instruction signals the IP lookup module 246 what to do 
next. For example, the instruction may signal the IP lookup 
module 246 to access an element in a second lookup table. 
Alternatively, the instruction may signal the IP lookup 
module 246 to use a DH contained within a particular 
DANET structure. If the entry in the first lookup array does 
not complete (i.e. identify a DANET structure to use) the 
process (see step 526 in FIG. 26), the receive ASIC 70 of 
FIG. 13 accesses an entry in the second lookup array and 
processes it (step 528 in FIG. 26). If the processing of this 
entry in the second lookup array does not complete the 
lookup (see step 530 in FIG. 26) the receive ASIC 70 access, 
an entry in the third lookup array and processes it (step 532 
in FIG. 26). If the instructions in the lookup arrays direct the 
use of an identified DANET structure in forwarding the 
packet, the receive ASIC 70 uses that structure (step 534 in 
FIG. 26). 

FIG. 28 depicts an example 570 that illustrates how the 
receive ASIC 70 uses the lookup arrays 564 in conjunction 
with the DANET structures 508 of FIG. 25. In the example 
570 depicted in FIG. 28, the 16-bit lookup array 572 
contains an entry 574 for the prefix 1.2/16. This entry 574 
advises the use of the 8-bit lookup array 576. The next bit in 
the destination address is then used to locate an entry, such 
as entry 582 or entry 584. Hie entry 582 is for the IP 
destination address 1.2.129/24. The DANET structure 586 is 
used in such an instance. For the IP destination address of 
1.2.128/17, the DANET structure 588 is used. 

The IP lookup is described in more detail in copending 
application entitled, "Network Packet Forwarding Lookup 
With AReduced Number Of Memory Accesses," application 
Ser. No. 09/237,128, filed on Jan. 25, 1999, which has been 
previously incorporated by reference. 

The IP lookup module 244 of FIG. 13 also performs 
policing of IP packets (see 1226 in FIG. 11). The IP lookup 
module 244 classifies IP packets into three bands: green, 
amber or red. Green implies that the traffic is within sus- 
tained rate traffic limits. Amber implies that the traffic is over 
the traffic limits, but under a predefined burst rate, and red 
implies that the traffic is over the burst rate. The policing 
may be used to mark the ToS bit 436 in the IP header 460 
of FIG. 23. In addition, the policer in the IP lookup module 
244 generates a profile indicator value in a range of one to 
four that is used as input to a Random Early Detect (RED) 
algorithm on the transmit ASIC 71. Each data flow has an 
associated traffic profile that sets limits on how much traffic 
the flow is allowed to generate. The flow limit is enforced by 
a token bucket algorithm that allows brief bursts above the 
flow limit. The token bucket assigns incoming traffic to the 
appropriate band. Thus, the IP lookup engine 244 performs 
both the policing function 1226 (FIG. 11) and the IP for- 
warding function (184 in FIG. 11). 

FIG. 29 is a flow chart 590 depicting the steps performed 
by the interconnect 62 as a part of the switching stage 84 of 
FIG. 6. The interconnect 62 redeems a ticket from the ticket 
master 232 of FIG. 13 to obtain data from the receive data 
parking lot 230 (step 592 in FIG. 29). The parking lot 230 
then transfers the data over the interconnect 62 (step 594 io 
FIG. 29). The interconnect 62 sends the data to the appro- 
priate line card, by way of example, fine card 53 of FIG. 4 
(step 596 in FIG. 29). The interconnect 62 then returns the 
ticket to the ticket master 232 on the receive ASIC 70 (step 
598 in FIG. 29). The interconnect 62 is described in more 
detail in copending application entitled, "Interconnect Net- 
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work For Operation Within A Communication Node," Ser. 
No. 09/336,090, which has been previously incorporated by 
reference. 

FIG. 30 is a functional block diagram of an illustrative 
interconnect card 62a of interconnect 62. The card 62<a 
includes an ASIC 720. As all of the interconnect cards are 
preferably identical, for the purpose of the following dis- 
cussion it is assumed that card 62a is an exemplary inter- 
connect card of a switching shelf 12. 

As shown in FIG. 30, the interconnect card 62a includes 
Gigabit transceiver sets 724 and 728, memory elements 730, 
controller 732 and status and control registers 734. Gigabit 
transceiver set 728 provides Gigabit I/O ports Qa-7a and 
06-76, which couple to the internal communication chan- 
nels of a switching shelf 12 of FIG. 2. Gigabit transceiver set 
724 provides Gigabit I/O ports 8a-15a and 86-156, which 
couple to the extended communication channels of the 
expansion shelf 18, shown in FIG. 1. 

Each transceiver of sets 724 and 728 couples to the ASIC 
720 by way of associated input and output shift and hold 
registers. More specifically, transceivers of set 724 couple to 
input shift and hold registers 734 by way of lines 736 and 
output shift and hold registers 738 by way of lines 740. 
Transceivers of set 724 couple to input shift and hold 
registers 742 by way of lines 744, and output shift and hold 
registers 746 by way of fines 748. 

The ASIC 720 also includes a dual-port RAM 750 for 
storing various stacks and queues 751 associated with flow 
control information. Flow status 753 stores an availability 
status, regarding the availability of a particular line card to 
receive information. RAM 750 intermediately stores infor- 
mation being transferred through the card 62a. Shift and 
hold registers 734 and 736 couple to the dual-port RAM 750 
by way of lines 752 and 754, respectively. Shift and hold 
registers 742 and 746 couple to the dual-port RAM 750 by 
way of lines 756 and 758. The dual-port RAM 750 also 
couples to destination stack 760 by way of lines 762. The 
ninety-six destination queues 760 intermediately store 
addresses representative of where particular data is stored in 
RAM 750. The queues 760, preferably employ a plurality of 
stacks for ease of addressing. However, other storage struc- 
tures can be employed. 

According to one preferred embodiment, the invention 
employs a plurality of memory storage queues/buffers to aid 
in the efficient transfer of information. It should be noted that 
the terms queue and buffer are used interchangeably. The 
dual-port RAM 750 provides an output queue for each 
transceiver of sets 724 and 726. More specifically, informa- 
tion cells coupled into card 62a to be transferred to a 
transmit ASIC 71 of FIG. 31, are first written into buffer 
memory at an address which is written into an output queue. 
Free list memory 762 provides a list of available buffer 
memory addresses. There is a reference counter 764 for each 
of the 1536 buffers in the dual port RAM 750. Reference 
counter 764 contains the number of output queues to which 
the contents of the respective buffers are to be sent. A 
reference counter 764 decrements in response to information 
being read from an associated buffer. When the reference 
counter reaches zero, the address of the buffer is returned to 
free list 745. In this way, the ASIC 720 can track the 
available buffer locations associated with each transceiver. 
Information written to buffer memory is subsequently trans- 
ferred to one of the output shift and hold registers 740 or 
748, and held there until an internal time slot arrives in 
which the destination address lookup can be performed, the 
read from the free list memory 762 can be performed, the 
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write to the buffer memory can be performed, and the write segments through the extension shelf 18, if an extension 

to the output queue can be performed. shelf 18 is included in the system. In the case of a commu- 

According to a preferred embodiment, the invention pro- nication node 10, configured as shown in FIG. 1, translation 

vides enhanced QoS features. To that end, queues 751 can memory 768 preferably contains nine logical storage areas; 

include QoS queues. The QoS queues, such as those con- 5 one for each switching shelf 12, and one for the extension 

ceptually illustrated in FIGS. 31A and 31B, can have mul- shelf 18. The extension shelf storage area is configured as a 

tiple watermark levels; those levels corresponding to differ- bitmap of destination line cards and priority. Destination 

ing priorities. By way of example, high-priority queue 800 address circuitry 770 accesses the translation memory 768, 

of FIG. 31Acan have two watermarks 806 and 808. In range ^ the mu iticast bitmap register 772 receives the accessed 

802, queue 800 reports its status as "stop-none," indicating 30 information. 

the I/O channel associated with queue 800 is ready to receive ." . . ACT „ . ,. - # „ 

. ' c • r* ■ • QOA Substantially identical ASICs are employed in the uiter- 

mformaUon of any priority. During operation, in range 804, 7 r J 

oaa - •♦ ♦ ♦ « .~ i™» :~A„ n t;„„ tu~ T/rt connect cards 60 of the switching shelves 12 as are 

queue 800 reports its status as stop-low, indicating the I/O . . UIf1fl ™ # , ACTP - in 

channel associated with queue 800 is ready to receive employed in the extensioc i shelf 18. To that ^en* ASIC 720 

information having a "medium" priority or higher. When the includes mode select 777 for selecting whether ASIC 740 is 

queue 800 is filled up to level 806, it reports its status as 15 t0 operate as an interconnection circuit or as an extended 

"stop-all." This indicates that its associated I/O channel is interconnection circuit. 

unavailable. If the "Stop-Low" watermark 808 of queue 800 Another feature of the illustrated ASIC 720 is a "slot 
has not been reached, it is available to receive information counter" contained in timers, counters, control registers 778. 
of any priority. The slot counter repeatedly counts from 0-15. Each port 
Low-priority queues, such as queue 810 depicted in FIG. 20 01-15a and 0b-15b is assigned a lot count. Each time the 
31 B, can include three watermarks 818, 820 and 822. Queue slot count 0-15 matches a port number, a check is performed 
810 reports a status of "Stop-None" in range 812. It reports to determine if there is a cell to be transmitted out that port, 
a status of "Stop-Low" in range 814. It reports a status of If there is, the cell is copied from RAM 750 to shift and hold 
"Stop-Medium" in range 816, and a status of "Stop-All" register 738 or 746 for transmission. If there is no cell to be 
subsequent to reaching watermark 818. 25 transmitted, then a flow control cell is transmitted. Accord- 
High-priority queues, such as queue 800, enable associ- ing to the illustrated embodiment, a common slot counter is 
ated line cards to pass low- and medium-priority traffic, employed for the a-ports and the b-ports. 
while not allowing low-priority traffic of one line card to As mentioned above, card 62a also includes controller 
strangle medium-priority traffic of a different line card. 732 and memory 730. Memory 730 stores the control code 
Low-priority queues, such as queue 810, enable associ- 30 for card 62a. As such, it provides start up initialization of 
ated line cards to pass low-, medium- and high-priority statuses, pointers and communication interfaces. Controller 
traffic, while not allowing low-priority and medium-priority 732 provides a variety of conventional processor functions, 
traffic of one line card to strangle high-priority traffic of a It should be noted that connections and circuit divisions 
different line card. It also prevents low-priority traffic of one ^ referred to in the above description may be representative of 
line card from strangling medium- and high-priority traffic both actual and logical connections or divisions, 
of a different line card. The interconnect 62 delivers the internal cells to the 
To efficiently manage information of differing priorities, transmit ASIC 64 of FIG. 5. The transmit ASIC 64 is 
the dual-port RAM 750 preferably provides storage for responsible for performing output processing (see 86 in FIG. 
sixty-four low-priority unicast queues; one for each possible 4Q 6) so the appropriate output data stream is output over the 
line card in the communication node 10. The RAM 750 also appropriate port. 

provides storage for sixteen high-priority unicast queues; pjo. 32 provides a more detailed flow diagram 600 

one for each line card of a switching shelf 12, one for each illustrative of output processing performed by the transmit 

potential additional switching shelf 12, and one extra queue. ASIC 71. As shown, the transmit parking lot 602 buffers 

Multicast traffic, preferably employs four low-priority and 45 output traffic from the interconnect 62. If the transmit ASIC 

four high-priority queues. 71 receives an internal cell as part of an IP packet, it defers 

Additionally, each plane of the extension shelf 18 output processing until all of the internal cells for that packet 

employs eight high-priority unicast queues; one for each are received. 

potential switching shelf 12. Each extension shelf logical piG. 33 depicts the transmit ASIC 71 in more detail. The 
plane also employs eight high-priority and eight low-priority 50 transmit ASIC 71 receives the 64-byte internal cell from the 
multicast queues; again, one for each potential switching interconnect 62. The transmit ASIC 71 removes the inter- 
shelf 12 destination. connect header 422 of FIG. 20, and sends the data portion 

A related component, the queue depth logic circuitry 766, 426 of the internal cell 420 to the transmit data parking lot 

maintains a status of all of the line cards of a switching shelf 610. The transmit data parking lot 610 may be implemented 

12. The status provides information regarding the availabil- 55 as an SDRAM. Those skilled in the art will appreciate that 

ity of each line card to receive information of varying the transmit data parking lot 610 may be implemented 

priority levels. alternatively with a number of other types of memory 

Another feature of the illustrated embodiment of the devices, 

invention is the way in which the node 10 passes the flow A ticket manager 612 manages the distribution of tickets, 

control status (sometimes referred to as back pressure status) 60 The ticket manager 612 has access to a ticket free list 

between the extension shelf 18 and each of the line cards of memory 614 and accesses the memory 614 to provide the 

the switching shelves 12. According to one preferred interconnect 62 with a free ticket pool 616 of locations in the 

embodiment, the invention utilizes bits of the internal transmit data parking lot 610 that are available for use. The 

canonical information cell, previously reserved for the des- interconnect 62 chooses one of the free tickets and presents 

tination address. 65 the ticket to the ticket manager 612. The interconnect 62 also 

The ASIC 720 also includes a translation memory 768. requests the data to be stored in the transmit data parking lot 

The translation memory 768 provides storage for path 610 at the location identified by the ticket 



10/27/2004, EAST Version: 1.4.1 



US 6,611,522 Bl 

29 30 

The interconnect 62 provides the ticket manager 612 with queues 622a-622h. If a cell or packet is part of a data flow 
the DH for the internal cell 420 and passes the DH to the cell requiring shaping, then the enqueuing and multicast corn- 
chain manager 618. The cell chain manager 618 accumulates ponent 628 passes the cell or packet through the calendar 
packets of cell chains. The cell chain manager 618 ensures queue 626. By using the calendar queue 626 only for traffic 
that all pieces (i.e. chunks of data) of an IP packet are s requiring shaping, the invention reduces the burden on the 
available before the IP packet is transmitted. enqueuing engine to reference and update information. The 

m . , , , .. f calendar queue 626 employs a logical nng structure with 

The output queue manager 620 provides scheduling for i ogica i s i ots 626a correspondmg to future moments on time, 
implementing the QoS features of the invenUon.lt manages ^ qlieue 626 has ^ enqlieU e and 
various output queues 622, which will be described in more dequeue pointers. The current time pointer advances accord- 
detail below. The output queue manager 620 cooperates with 10 t0 a ^ mt schedule based on the width of a slot 626a in 
a QoS table 624 and a calendar queue 626. the calendar ring. The enqueue pointer points to the slot that 

The output data stream need not be a unicast data stream, the data is being scheduled into, and the dequeue pointer 

but rather may be a multicast data stream such that the same points to the slot from which data is being dequeued from 

data stream is sent to multiple destinations. The enqueuing the calendar queue. Data is queued based on a desired 

and multicast component 628 in FIG. 33 is responsible for 15 transmit rate, such that a "future time" is calculated for the 

both in enqueuing cells in the transmit queues 622 and item to be queued, based on the last transmit time of an item 

performing steps necessary to support multicast output. ™ a particular data flow. The "future time" cannot be less 

Multicast packets or cells are identified by the enqueuing toe slot painted to by the current Ume pointer. The 

and multicast component 628, and given a multicast iden- 'akndar ™? hcs on tablc ™ to £ hc ^ 

tifier that corresponds to an ATM or IP multicast group. The 20 ? a ' a m £ * e cak * dar t *> c ™ :™ ' "PP"^* ™ e QoS 

, v u . t t ~ 0 r ♦ ,u i table 624 stores indicators of the QoS features required by 

enqueuing and multicast component 628 replicates the pack- ^ m ^ d ^ ^ 1q{ 61Q p^^ly, me cp 

ets or cells to be sent to generate as many copies as there are fi4 Qf nGS 4 ^ 9 ^ ^ ^ m the Lcp 

destinations specified in a multicast alias table 630. The ^ 0 f FIGS 5 and 9. 

enqueuing and multicast component 628 transfers the rep- Jhe dequeue pf0cess for tfae caletldar qiieue 626 i s 

heated data into the appropriate output queues 622. asynchronous relative to the enqueue process. The dequeue 

FIG. 34 provides a more detailed block diagram of the process removes all entries for the slot pointed to by the 

queuing structure 700 of the transmit ASIC 71. As shown in dequeue pointer and advances dequeue pointer until it 

FIGS. 33 and 34, the queuing structure 700 includes a catches up with the current time pointer. The entries 

calendar queue 626 and output queues 622. As shown, the 3Q removed from the "dequeue slot" are placed into the output 

output queues 622 and the calendar queue 626 receives data queues 622a-622/t specified by their QoS features. As 

from the enqueuing and multicast component 628, Accord- shown in FIG. 34, data that is not subjected to shaping 

ing to the illustrative embodiment, each output port includes passes directly to the output queues 622a-622/i. 

eight output queues 622a-622/i. The DH specifies in which Alternatively, data that is subject to shaping is placed in the 

queue to put the cell. The following provides an example 35 calendar queue for 626 until dequeued 632. 

usage of the queues. A queue scheduler shown at 604 in FIG. 32 (in the output 

The interrupt queue 622A is the highest priority queue and queue manager 620) dequeues data from the output queues 
is dequeued immediately. The interrupt queue 622/z is used 622a-622A, also shown in FIG. 32. The scheduler 604 
for extremely urgent data that has to be transmitted ahead of implements both priority and weighted round robin sched- 
other information. Priority queues 622a-622g are for dif- 40 uling. A programmable threshold divides priority queues 
ferent priorities of data. These priority queues 622a-622g from weighted round robin queues. The scheduler 604 first 
are serviced in accordance with a weighted round robin processes the priority queues, transmitting traffic in strict 
scheme wherein the data in the higher priority queues (e.g. priority order. The rest of the queues 622 are processed in 
priority one queue 622g) is serviced prior to the servicing of weighted round robin order. Each output queue is typically 
lower priority queues (e.g. priority five queue 622c). The 45 assigned to QoS class, with the weighted priorities on the 
best effort queue 622b is used for traffic that has no guar- queues configured accordingly. The priority threshold can be 
antees or assurances of delivery. The less effort queue 622a used to select priority queuing only or weighted round robin 
is used for data that has been tagged as being in the violation queuing only for all of the output queues 622a-622h. 
of a service contract, and can be dropped if there is not Additionally, output queues 622<z-622/i of one logical out- 
sufficient available bandwidth. In general, data on the less 50 put port can be configured independently from output 
effort queue 622a is not expected to be transmitted, but can queues 622a-622h of any other logical output port, 
be if there is available bandwidth. As skilled artisans will fig. 35 is a more detailed diagram 710 illustrative of the 
appreciate, the output queue structure of FIG. 34 is consid- dequeuing process of the output queues 622 into an output 
ered to be illustrative in nature, and any number of output FIFO 712. Threshold levels in the output FIFO 712 trigger 
queue structures may be employed without impacting the 55 dequeuing. The output FIFO 712 triggers additional data to 
scope of the present invention. be dequeued in response to having its content fall below a 

The queuing structure 700 also monitors the amount of selected level. The weighting of the priority queues 

data stored in each queue. If the amount of data in a queue 622c-622g allows a user to specify how much bandwidth 

exceeds a certain threshold, congestion control may be can be consumed by a particular queue, and prevents a 

performed (e.g., PPD, EPD and RED) to discard or mark go higher priority queue from consuming all of the available 

traffic destined to the queue. Information about the discarded bandwidth. According to the illustrative embodiment, the 

traffic may also be sent to the control processor to identify queuing structure achieves this by only dequeuing a maxi- 

those flows that are contributing most to the output conges- mum amount of data allowed by the weight from a particular 

tion so that penalty actions, such as previously discussed, queue, prior to moving on to dequeue data from a lower 

may be performed on those flows. $s priority queue with the illustrative queuing structure. 

The calendar queue 626 shapes or rate limits traffic. The According to the illustrative embodiment, each commu- 

calendar queue 626 regulates the data to be placed into the nication module 12 of FIG. 2 includes up to 48 logical output 
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ports and each logical output ports has an associated set of 
output queues 622. The following describes an example 
method of using the queues. 

The queue 622Jt is an interrupt queue. As mentioned 
above, this queue is serviced first, and it is used for 
extremely urgent data that has to be transmitted ahead of any 
other information. Typical uses include link management 
frames or cells or delay sensitive traffic that cannot be 
queued any other way. In the illustrative embodiment, this 
queue 622h is used internally by the communication node 
10, and only when absolutely necessary. 

The queues 622c-622g are priority queues. These queues 
are for any traffic that is treated better than best-effort and 
has rate limits. Data from the calendar queue 626 is placed 
into these queues according to priority. For example, the 
queue manager 620 places extremely time sensitive CBR 
and rtVBR traffic into the Priority 1 queue 622g, while it 
places nrt VBR traffic into a lower priority queue, such as the 
Priority 5 queue 622e. 

The queue 6226 is a "best effort" queue. This is the queue 
that is used for traffic that has no associated QoS guarantees 
or assurances of delivery. Typically, the queue manager 
directs UBR and non-reserved IP traffic into the queue 622b, 

The queue 622a provides a "less effort" queue. The queue 
622a can be used for data that is tagged as being in violation 
of a traffic service contract or would be dropped if there is 
not any available bandwidth. Data in the queue 622a is not 
expected to be transmitted, but can be if there is available 
bandwidth. Another use for this queue is for "misbehaving" 
best effort flows. For example, if a flow is experiencing 
excessive discards from the RED algorithm, the flow can be 
classified and placed in the "less effort" queue as a penalty. 

The output queue manager 620 passes a ticket list and a 
DH to the encapsulation selector 632. The encapsulation 
selector 632 then retrieves the appropriate data from the 
transmit parking lot 610. The encapsulation selector 632 
passes the destination handle for the selected cells to the 
destination description manager 634. The destination 
description manager 634 works in conjunction with the 
encapsulation engine 636 to determine how to appropriately 
encapsulate the data that is to be output. The destination 
description manager 634 accesses the encapsulation RAM 
638 to obtain information regarding the appropriate encap- 
sulation for the destination. The destination description 
manager 634 has a destination descriptor for the destination 
of the output data stream. The DH, which accompanies 
every cell, is used by the destination description manager 
634 to locate a destination descriptor. The destination 
descriptor is a field found within the DH and contains all of 
the information necessary for reencapsulating the cell, 
including partial cyclic redundancy checks and information 
regarding the length of the frame. The encapsulation engine 
634 uses an encapsulation identifier from the destination 
descriptor engine 640 to reference a table of encapsulation 
descriptors 642. The encapsulation descriptor from table 642 
contains a pattern to be inserted into the beginning of an 
outgoing frame that identifies the type of encapsulation. 

The encapsulation engine 636 gathers the DH and the data 
retrieved from the transmit data parking lot 610, packages 
the data in the appropriate encapsulation and forwards the 
data for ATM output 644. The ATM output module 644 
creates a correct AAL5 trailer and sets various bits in the 
cell. The Operations Administration and Maintenance 
(OAM) element 646 provides operation and control func- 
tions within the ATM protocol set. The ATM output module 
644 transmits the resulting data to the PLCP module 648. If 
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no PLCP encapsulation is required, the cells pass through 
the PLCP module 648 to the port transmit queue 622, 
without modification. Otherwise, the PLCP module 648 
encapsulates the cells into PLCP frames. 

The encapsulation engine 636 passes IP packets to the 
PPP/FR output module 650, which PPP frames or FR frames 
the IP packets for encapsulation. The PPP/FR output module 
650 passes the resulting frames to the port transmit queues 
622. The encapsulation engine 636 passes certain packets to 
the LCP 72 of FIG. 5 by way of the LCP packet output 
module 652 and the LCP buffer 654. 

A SONET framer/physical interface 656 frames the data 
into SONET frames and performs parallel to serial conver- 
sion. The SONNET framer/physical interface 656 is a physi- 
cal interface to the output lines. The resulting data is output 
towards its destination. 

Thus, the illustrative embodiment of the invention pro- 
vides a QoS facility for operation within a digital commu- 
nication node. The digital communication node is essentially 
a single device and can forward IP packets, as well as ATM 
cells. The illustrative QoS facility provides QoS features for 
both IP-based data streams and ATM-based data streams. 
Since the illustrative communication node transfers data 
internally in an internal canonical form, it can be easily 
adapted to forward and apply QoS features to data streams 
using, substantially, any encapsulation scheme. 

While the present invention has been described with 
reference to an illustrative embodiment thereof, those skilled 
in the art will appreciate the various changes in form and 
detail may be made without departing from the intended 
scope of the present invention as in the appended claims. 

Having described the invention, what is claimed as new 
and protected by Letters Patent is: 

1. A facility for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said facility com- 
prising: 

a plurality of logical input ports adapted for receiving 
input data flows from external data sources and a 
plurality of logical output ports adapted for transmitting 
output data flows to a plurality of external data 
destinations, wherein said input data flows and said 
output data flows include a plurality of ATM data cells 
and a plurality of IP data packets; 

ATM forwarding means for forwarding ATM data cells 
from one of said logical input ports toward at least one 
of said logical output ports along a selected forwarding 
path; 

IP forwarding means for forwarding IP data packets from 
one of said logical input ports toward at least one of 
said logical output ports along a selected forwarding 
path; 

QoS elements for identifying one or more ATM QoS 
features for ATM data cells in the input data flows, 
identifying one or more IP QoS features for IP data 
packets in the input data flows, and scheduling for- 
warding of said input data flows, based at least in part, 
on the identified one or more ATM QoS features and on 
the identified one or more IP QoS features; and 

a housing that contains both said ATM forwarding means 
and said IP forwarding means. 

2. A facility for providing ATM and IP QoS features 
according to claim 1, wherein the housing also includes said 
QoS elements. 

3. A facility for providing ATM and IP QoS features 
according to claim 1, wherein said QoS elements further 
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comprise call control means for responding to service con- 
tract requests from at least one of said external data sources 
and said external data destinations, and for selectively 
forming service contracts between said communication node 
and at least one of said external data sources and said 
external data destinations, wherein said service contracts 
include agreements by said communication node to provide 
said ATM and IP QoS features to at least one of input data 
flows from said external data sources and input data flows 
directed toward said external data destinations, and wherein 
said communication node is adapted for providing differing 
service contracts to different ones of said external data 
sources and said external data destinations. 

4. A facility for providing ATM and IP QoS features 
according to claim 3, wherein said call control means is 
adapted for determining an available bandwidth of said 
communication node and for denying a service contract 
request in response to determining that insufficient band- 
width is available to provide a requested QoS feature. 

5. A facility for providing ATM and IP QoS features 
according to claim 3, wherein said QoS elements further 
comprise traffic control means, responsive to said call con- 
trol means, for interpreting said ATM and IP QoS features 
provided by said service contracts, and for signaling control 
information to devices along said forwarding paths to ensure 
adequate bandwidth along said forwarding paths to provide 
said ATM and IP QoS features commensurate with said 
service contracts. 

6. A facility for providing ATM and IP QoS features 
according to claim 1, wherein said QoS elements further 
comprise ReSerVation Protocol (RSVP) means for provid- 
ing a signaling mechanism for an external data destination to 
request a service contract from said communication node for 
data flows directed toward said external data destination. 

7. A facility for providing AIM and IP QoS features 
according to claim 1, wherein said QoS elements further 
comprise data classification means for selecting for particu- 
lar QoS categories, IP data packets and ATM data cells, 
which have been input to said communication node, by 
determining which QoS features, if any, are required by said 
input IP data packets and said input ATM data cells. 

8. A facility for providing ATM and IP QoS features 
according to claim 7, wherein said QoS elements farther 
comprise scheduling means for scheduling forwarding of 
said IP data packets and switching of said ATM data cells in 
response to said QoS categories selected by said classifica- 
tion means. 

9. A facility for providing ATM and IP QoS features 
according to claim 1, wherein said QoS elements further 
comprise data policing means for determining if said IP data 
packets and said ATM data cells are part of a particular one 
of said input data flows, for determining if an external source 
of said particular data flow has a service contract with said 
communication node, and for determining whether said IP 
data packets and said ATM data cells of said particular data 
flow are in accord with said service contract. 

10. A facility for providing ATM and IP QoS features 
according to claim 9, wherein said policing means further 
includes discarding means for discarding, selected ones of 
said IP data packets and said ATM data cells, which are not 
in accord with said service contract. 

11. A facility for providing ATM and IP QoS features 
according to claim 9, wherein said policing means further 
includes marking means for marking, selected ones of said 
IP data packets and said ATM data cells, which are not in 
accord with said service contract. 

12. A facility for providing ATM and IP QoS features 
according to claim 9, wherein one or more of said ATM data 
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cells make up an ATM frame, and wherein said policing 
means further includes Partial Packet Discard (PPD) means 
for discarding selected additional ATM data cells in an ATM 
frame in response to said policing means determining that 
another of said ATM data cells is not in accord with a service 
contract 

13. A facility for providing ATM and IP QoS features 
according to claim 9, wherein one or more of said ATM data 
cells make up an ATM frame, and wherein said policing 
means further includes ATM queuing means for buffering 
said forwarding of ATM data cells and Early Packet Discard 
(EPD) means for discarding entire ATM frames in response 
to said ATM queuing means reaching a selected level of 
fullness. 

14. A facility for providing ATM and IP QoS features 
according to claim 9, wherein said policing means further 
includes queuing means for buffering said forwarding of 
ATM data cells and for buffering said forwarding of said IP 
data packets, and Random Early Detect means for substan- 
tially randomly discarding IP data packets and ATM data 
cells in response to said queuing means reaching one or 
more selected levels of fullness. 

15. A facility for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said facility com- 
prising: 

a plurality of logical input ports adapted for receiving 
input data flows from external data sources and a 
plurality of logical output ports adapted for transmitting 
output data flows to a plurality of external data 
destinations, wherein said input data flows and said 
output data flows include a plurality of ATM data cells 
and a plurality of IP data packets; 

ATM forwarding means for forwarding ATM data cells 
from one of said logical input ports toward at least one 
of said logical output ports along a selected forwarding 
path; 

IP forwarding means for forwarding IP data packets from 
one of said logical input ports toward at least one of 
said logical output ports along a selected forwarding 
path; 

QoS elements for identifying one or more ATM QoS 
features for ATM data cells in the input data flows, 
identifying one or more IP QoS features for IP data 
packets in the input data flows, and scheduling for- 
warding of said input data flows, based at least in part, 
on the identified one or more ATM QoS features and on 
the identified one or more IP QoS features; and 

an Application Specific Integrated Circuit that contains at 
least a portion of said ATM forwarding means, said IP 
forwarding means, and said QoS elements. 

16. A facility for providing ATM and IP QoS features 
according to claim 15 further comprising a common physi- 
cal interface that includes said logical input ports from 
which said ATM forwarding means receives said ATM data 
cells and said IP forwarding means receives said IP data 
packets. 

17. A facility for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said facility com- 
prising: 

a plurality of logical input ports adapted for receiving 
input data flows from external data sources and a 
plurality of logical output ports adapted for transmitting 
output data flows to a plurality of external data 
destinations, wherein said input data flows and said 
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output data flows include a plurality of ATM data cells 
and a plurality of IP data packets, and said input data 
flows include Synchronous Optical Network (SONET) 
frames; 

ATM forwarding means for forwarding ATM data cells 
from one of said logical input ports toward at least one 
of said logical output ports along a selected forwarding 
path; 

IP forwarding means for forwarding IP data packets from 
one of said logical input ports toward at least one of 
said logical output ports along a selected forwarding 
path; 

QoS elements for identifying one or more ATM QoS 
features for ATM data cells in the input data flows, 
identifying one or more IP QoS features for IP data 
packets in the input data flows, and scheduling for- 
warding of said input data flows, based at least in part, 
on the identified one or more ATM QoS features and on 
the identified one or more IP QoS features; and 

a SONET deframer for deframing said SONET frames in 
said input data flows. 

18. A facility for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said facility com- 
prising: 

a plurality of logical input ports adapted for receiving 
input data flows from external data sources and a 
plurality of logical output ports adapted for transmitting 
output data flows to a plurality of external data 
destinations, wherein said input data flows and said 
output data flows include a plurality of ATM data cells 
and a plurality of IP data packets; 

ATM forwarding means for forwarding ATM data cells 
from one of said logical input ports toward at least one 
of said logical output ports along a selected forwarding 
path, wherein said ATM forwarding means includes 
ATM lookup means for identifying toward which of 
said logical output ports to forward said ATM data cells 
in said input data flows based on information contained 
in said ATM data cells; 

IP forwarding means for forwarding IP data packets from 
one of said logical input ports toward at least one of 
said logical output ports along a selected forwarding 
path; and 

QoS elements for identifying one or more ATM QoS 
features for ATM data cells in the input data flows, 
identifying one or more IP QoS features for IP data 
packets in the input data flows, and scheduling for- 
warding of said input data flows, based at least in part, 
on the identified one or more ATM QoS features and on 
the identified one or more IP QoS features. 

19. A facility for providing ATM and IP QoS features 
according to claim 18, wherein said ATM lookup means is 
further adapted for determining which of said ATM QoS 
features should be applied to said ATM data cells in said 
input data flows. 

20. A facility for providing ATM and IP QoS features 
according to claim 19, wherein said ATM lookup means is 
further adapted for performing said identifying and said 
determining in a single lookup operation. 

21. A facility for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said facility com- 
prising: 

a plurality of logical input ports adapted for receiving 
input data flows from external data sources and a 
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plurality of logical output ports adapted for transmitting 
output data flows to a plurality of external data 
destinations, wherein said input data flows and said 
output data flows include a plurality of ATM data cells 
and a plurality of IP data packets; 
ATM forwarding means for forwarding ATM data cells 
from one of said logical input ports toward at least one 
of said logical output ports along a selected forwarding 
path; 

IP forwarding means for forwarding IP data packets from 
one of said logical input ports toward at least one of 
said logical output ports along a selected forwarding 
path, wherein said IP forwarding means includes IP 
lookup means for identifying toward which of said 
logical output ports to route said IP data packets in said 
input data flows based on address information con- 
tained in said IP data packets; and 

QoS elements for identifying one or more ATM QoS 
features for ATM data cells in the input data flows, 
identifying one or more IP QoS features for IP data 
packets in the input data flows, and scheduling for- 
warding of said input data flows, based at least in part, 
on the identified one or more ATM QoS features and on 
the identified one or more IP QoS features. 

22. A facility for providing ATM and IP QoS features 
according to claim 21, wherein said IP lookup means is 
further adapted for determining which of said IP QoS 
features should be applied to said IP data packet. 

23. A facility for providing AIM and IP QoS features 
according to claim 1, wherein said ATM QoS features 
include at least one of, Constant Bit Rate (CBR), Unspeci- 
fied Bit Rate (UBR), non-real-time Variable Bit Rate 
(nrtVBR), real-time Variable Bit Rate (rtVBR) and Available 
Bit Rate (ABR), and said IP QoS features include at least 
one of, Provisioned QoS, Differentiated Services, and Inte- 
grated Services. 

24. A facility for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said facility com- 
prising: 

a plurality of logical input ports adapted for receiving 
input data flows from external data sources, and a 
plurality of logical output ports adapted for transmitting 
output data flows to a plurality of external data 
destinations, wherein said input data flows and said 
output data flows include at least one of a plurality of 
ATM data cells and a plurality of IP data packets; and 

a plurality of communication modules, wherein said com- 
munication modules include: 

IP packet forwarding elements for forwarding IP data 
packets from one of said logical input ports toward 
at least one of said logical output ports, 

ATM cell forwarding elements for forwarding ATM 
data cells from one of said logical input ports to at 
least one of said logical output ports, and 

QoS elements adapted for identifying one or more ATM 
QoS features associated with the ATM data cells in 
the input data flows, identifying one or more IP QoS 
features associated with the IP data packets in the 
input data flows, and providing the identified one or 
more ATM QoS features and the identified one or 
more IP QoS features to said input data flows in said 
communication node, wherein at least some of said 
QoS elements are distributed in said plurality of 
communication modules. 

25. A facility for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said facility com- 
prising: 
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a plurality of logical input ports adapted for receiving features associated with the IP data packets in the 

input data flows from external data sources, and a input data flows, and providing the identified one or 

plurality of logical output ports adapted for transmitting more ATM QoS features and the identified one or 

output data flows to a plurality of external data more IP QoS features to said input data flows in said 

destinations, wherein said input data flows and said 5 communication node, and 

output data flows include at least one of a plurality of lookup engines adapted for processing IP data packets 

ATM data cells and a plurality of IP data packets; and ATM data cells in an input data flow to determine 

a plurality of communication modules, wherein said com- ATM and IP QoS features required by said IP data 

munication modules include: packets and said ATM data cells in said input data 

IP packet forwarding elements for forwarding IP data 10 flow, 

packets from one of said logical input ports toward 31. A facility for providing AIM and IP QoS features 

at least one of said logical output ports, according to claim 30, wherein said lookup engines are 

ATM cell forwarding elements for forwarding ATM further adapted for processing said IP data packets and said 

data cells from one of said logical input ports to at ATM data cells in an input data flow to identify one or more 

least one of said logical output ports, and 35 of said logical output ports towards which said ATM data 

QoS elements adapted for identifying one or more ATM cells and said IP data packets should be forwarded. 

QoS features associated with the ATM data cells in 32. A facility for providing ATM and IP QoS features 

the input data flows, identifying one or more IP QoS according to claim 31, wherein said lookup engines are 

features associated with the IP data packets in the further adapted for generating a destination handle repre- 

input data flows, and providing the identified one or 2Q sentative of said QoS features required by said ATM data 

more ATM QoS features and the identified one or cells and said IP data packets, and representative of said one 

more IP QoS features to said input data flows in said or more logical output ports towards which said ATM data 

communication node; and cells and said IP data packets are to be forwarded, 

an interconnect in digital communication with said com- 33, A facility for providing ATM and IP QoS features 

munication modules and adapted for forwarding ATM 2 5 according to claim 31, wherein said lookup engines are 

data cells and IP data packets between said communi- further adapted for determining said QoS features and said 

cation modules. logical output ports in common lookup operations. 

26. A facility for providing ATM and IP QoS features 34. A facility for providing ATM and IP QoS features 
according to claim 25, wherein at least some of said QoS according to claim 30, wherein said ATM data cells include 
elements are distributed in said interconnect. 30 ATM cell headers and said lookup engines are further 

27. A facility for providing ATM and IP QoS features adapted for processing said ATM cell headers to determine 
according to claim 24, further comprising an interconnect in ATM QoS features required by said ATM cells in an input 
digital communication with said communication modules data flow. 

and adapted for forwarding ATM data cells and routing IP 35. A facility for providing ATM and IP QoS features 

data packets between said communication modules, wherein 35 according to claim 34, wherein said ATM cell headers 

at least some of said QoS elements are distributed in said include Virtual Circuit Indicators (VCIs) and Virtual Path 

interconnect. Indicators (VPIs) and said lookup engines are further 

28. A facility for providing ATM and IP QoS features adapted to process said VCIs and VPIs to determine ATM 
according to claim 24 further comprising a control processor QoS features required by said ATM cells in an input data 
for controlling operation of said QoS elements. 40 flow. 

29. A facility for providing ATM and IP QoS features 36. A facility for providing ATM and IP QoS features 
according to claim 28, wherein said communication mod- according to claim 30, wherein said IP data cells include IP 
ules include communication module processors that are in cell headers comprising at least one of, destination address, 
communication with said control processor and assist said source address, IP protocol number, input port number, 
control processor in controlling said QoS elements, 45 output port number and source Autonomous System (AS), 

30. A facility for providing Asynchronous Transfer Mode and wherein said lookup engines are further adapted for 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) processing said IP cell headers to determine QoS features 
features in a digital communication node, said facility com- required by said IP data packets in an input data flow, 
prising: 37. A facility for providing Asynchronous Transfer Mode 

a plurality of logical input ports adapted for receiving 50 (ATM) and Internet Protocol (IP) Quality of Service (QoS) 
input data flows from external data sources, and a features in a digital communication node, said facility corn- 
plurality of logical output ports adapted for transmitting prising: 

output data flows to a plurality of external data a plurality of logical input ports adapted for receiving 

destinations, wherein said input data flows and said input data flows from external data sources, and a 

output data flows include at least one of a plurality of 55 plurality of logical output ports adapted for transmitting 

ATM data cells and a plurality of IP data packets; and output data flows to a plurality of external data 

a plurality of communication modules, wherein said com- destinations, wherein said input data flows and said 

munication modules include: output data flows include at least one of a plurality of 

IP packet forwarding elements for forwarding IP data ATM data cells and a plurality of IP data packets; and 

packets from one of said logical input ports toward 60 a plurality of communication modules, wherein said com- 

at least one of said logical output ports, munication modules include: 

ATM cell forwarding elements for forwarding ATM IP packet forwarding elements for forwarding IP data 

data cells from one of said logical input ports to at packets from one of said logical input ports toward 

least one of said logical output ports, at least one of said logical output ports, 

QoS elements adapted for identifying one or more ATM 65 ATM cell forwarding elements for forwarding ATM 

QoS features associated with the ATM data cells in data cells from one of said logical input ports to at 

the input data flows, identifying one or more IP QoS least one of said logical output ports, 
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QoS elements adapted for identifying one or more ATM 
QoS features associated with the ATM data cells in 
the input data flows, identifying one or more IP QoS 
features associated with the IP data packets in the 
input data flows, and providing the identified one or 
more AIM QoS features and the identified one or 
more IP QoS features to said input data flows in said 
communication node, and 

policing elements for detecting if said IP data packets 
and said ATM data cells in an input data flow exceed 
a selected QoS feature. 

38. A facility for providing ATM and IP QoS features 
according to claim 30, wherein said communication mod- 
ules include policing elements for detecting nonconforming 
IP data packets and nonconforming ATM data cells in an 
input data flow, wherein said nonconforming IP data packets 
and ATM data cells exceed a selected QoS feature. 

39. A facility for providing ATM and IP QoS features 
according to claim 38, wherein said lookup engine includes 
at least some of said policing elements, and is further 
adapted for marking said nonconforming ATM data and IP 
data packets. 

40. A facility for providing AIM and IP QoS features 
according to claim 39, wherein said lookup engines are 
further adapted for generating destination handles represen- 
tative of said QoS features required by said ATM data cells 
and said IP data packets, and for performing said marking of 
said nonconforming ATM data cells and said IP data packets 
by setting bits in said destination handle. 

41. A facility for providing ATM and IP QoS features 
according to claim 25, wherein 

said communication modules further comprise lookup 
engines adapted for processing IP data packets and 
ATM data cells in an input data flow to determine AIM 
and IP QoS features required by said IP data packets 
and said AIM data cells in said input data flow, and for 
generating destination handles representative of said 
QoS features required by said AIM data cells and said 
IP data packets, and 

said interconnect further comprises input data queues for 
IP data packets and ATM data cells, wherein said 
interconnect determines which of said input data 
queues to store particular ones of said IP data packets 
and said AIM data cells based at least in part on said 
IP and AIM QoS features indicated by said destination 
handle. 

42. A facility for providing AIM and IP QoS features 
according to claim 25 wherein AIM data cells and IP data 
packets transferred from said interconnect to one of said 
communication modules include an associated status indi- 
cating an ability of one or more of others of said commu- 
nication modules to receive additional AIM data cells and IP 
data packets. 

43. A facility for providing AIM and IP QoS features 
according to claim 25, wherein said communication mod- 
ules further include a queuing structure for intermediately 
storing AIM data cells and IP data packets transferred from 
said interconnect to said communication modules for output 
via one or more of said logical output ports. 

44. A facility for providing AIM and IP QoS features 
according to claim 43, wherein said communication mod- 
ules include one or more physical output ports associated 
with each of said logical output ports, said queuing structure 
further comprises a plurality of output queues associated 
with each of said physical output ports, wherein each 
plurality of output queues is adapted for intermediately 
storing ones of said IP data packets and AIM data cells 
destined for output via said associated physical output port. 
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45. A facility for providing AIM and IP QoS features 
according to claim 44, wherein output queues included in a 
particular plurality of output queues have an assigned pri- 
ority relative to other output queues included in said par- 
ticular plurality, wherein data stored in an output queue 
having a relatively higher priority is scheduled for output in 
preference to data stored in a queue having a relatively lower 
priority. 

46. A facility for providing AIM and IP QoS features 
according to claim 44, wherein said queuing structure fur- 
ther includes a calendar queue, wherein said calendar queue 
is adapted for intermediately storing at least selected ones of 
said AIM data cells and IP data packets destined for said 
output queues, and for scheduling transfer of said selected 
ones of said AIM data cells and IP data packets from said 
calendar queue to said output queues based at least in part on 
which of said AIM and IP QoS features apply to said 
selected ones of said AIM data cells and IP data packets. 

47. A facility for providing AIM and IP QoS features 
according to claim 45, wherein said queuing structure fur- 
ther comprises an output stack, wherein said output stack is 
adapted for intermediately storing said AIM data cells and 
IP data packets destined for transfer from one of said 
pluralities of output queues, and said facility further com- 
prises a processor for transferring said AIM data cells and 
IP data packets from said plurality of output queues to said 
output stack according to a selected priority, wherein said 
selected priority is based at least in part on from which one 
of said output queues said data being transferred. 

48. A facility for providing ATM and IP QoS features 
according to claim 15, wherein said QoS elements further 
comprise means for providing a static QoS contract between 
said communication node and an external device. 

49. A facility for providing AIM and IP QoS features 
according to claim 1 wherein the QoS elements are further 
configured to prioritize transfer of said AIM data cells and 
IP data packets through said facility based at least in part on 
the identified one or more AIM QoS features and the 
identified one or more IP QoS features. 

50. A facility for providing AIM and IP QoS features 
according to claim 18, further comprising: 

AIM prioritizing elements for prioritizing transfer of said 
AIM data cells from said logical input ports to said 
logical output ports, based at least in part, on the 
identified one or more ATM QoS features. 

51. A facility for providing AIM and IP QoS features 
according to claim 21, further comprising: 

IP prioritizing elements for prioritizing transfer of said IP 
data packets from said logical input ports to said logical 
output ports, based at least in part, on the identified one 
or more IP QoS features. 

52. A method for providing Asynchronous Transfer Mode 
(AIM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said method 
comprising: 

receiving, by a plurality of logical input ports, input data 
flows from external data sources, wherein said input 
data flows include a plurality of ATM data cells and a 
plurality of IP data packets; 

transmitting, by a plurality of logical output ports, output 
data flows to a plurality of external data destinations, 
wherein said output data flows include a plurality of 
ATM data cells and a plurality of IP data packets; 

identifying one or more of said logical output ports to 
which to forward said AIM data cells in said input data 
flows based on information contained in said AIM data 
cells; 
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forwarding ATM data cells from one of said logical input 
ports toward the identified one or more logical output 
ports along a selected forwarding path; 

forwarding IP data packets from one of said logical input 
ports toward at least one of said logical output ports 5 
along a selected forwarding path; 

identifying one or more AIM QoS features for ATM data 
cells in the input data flows; 

identifying one or more IP QoS features for IP data 
packets in the input data flows; and 

scheduling forwarding of said input data flows, based at 
least in part, on the identified one or more ATM QoS 
features and on the identified one or more IP QoS 
features. 15 

S3. A method for providing Asynchronous Transfer Mode 
(ATM) and Internet Protocol (IP) Quality of Service (QoS) 
features in a digital communication node, said method 
comprising: 

receiving, by a plurality of logical input ports, input data 20 
flows from external data sources, wherein said input 
data flows include a plurality of ATM data cells and a 
plurality of IP data packets; 

transmitting, by a plurality of logical output ports, output 
data flows to a plurality of external data destinations, 



wherein said output data flows include a plurality of 
ATM data cells and a plurality of IP data packets; 
identifying one or more of said logical output ports to 
which to route said IP data packets in said input data 
flows based on address information coutained in said IP 
data packets; 

forwarding ATM data cells from one of said logical input 

ports toward at least one of said logical output ports 

along a selected forwarding path; 
forwarding IP data packets from one of said logical input 

ports toward the identified one or more logical output 

ports along a selected forwarding path; 
identifying one or more ATM QoS features for ATM data 

cells in the input data flows; 
identifying one or more IP QoS features for IP data 

packets in the input data flows; and 
scheduling forwarding of said input data flows, based at 

least in part, on the identified one or more ATM QoS 

features and on the identified one or more IP QoS 

features. 
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